Data Engineering Mastermind

Critical Techworks; Braga; On-site; EUR 45,000 - 65,000; Full time

Job summary

Critical TechWorks in Portugal is seeking a Data Engineer to help shape the data landscape for automotive innovation. You will design and maintain data pipelines, model data, ensure quality, and optimize systems across large-scale data platforms.

You’ll work with AWS, Terraform, SQL and multiple programming languages (Python/Java/JavaScript/Scala), plus Spark, Hadoop, Hive, Cassandra, Airflow to deliver real-time insights.

Qualifications

• At least 3 years of experience in data engineering.
• Experience with public cloud platforms, especially AWS.
• Experience with IaC tools, such as Terraform.
• Ability to code in Python, Java, JavaScript, or Scala.
• Experience with data-driven architectures, preferably in public cloud platforms.
• Experience using SQL for data querying.
• Experience with Big Data and data modelling.
• Advanced knowledge of multiple SQL/noSQL environments and a solid rationale for tool selection.
• Familiarity with Big Data tools (Hadoop, Cassandra, Hive, Spark, etc.).
• Experience with Airflow is a big plus.

Responsibilities

• Data Pipeline Development: design and maintain robust data pipelines.
• Data Modeling: craft and optimize data models for efficient retrieval.
• Data Quality Assurance: implement testing and validation for data accuracy.
• Troubleshooting and System Optimization: identify and solve data processing issues.
• Big Data Processing: work on large-scale data projects using Spark.
• Real-Time Data Streaming: build real-time data processing and insights.

Skills

Data Pipeline

Data Modeling

Data Quality

Troubleshooting

Big Data

Real-Time Streaming

Tools

Apache Spark

Terraform

SQL

Hadoop

Cassandra

Apache Hive

Airflow

Job description

We’re looking for bright Data Engineer Mastermindsto join our team

At Critical TechWorks, we're not just engineering software; we're sculpting the data landscape of tomorrow. Our Data Engineers are pivotal in transforming the BMW Group into a data-driven powerhouse. From handling vast datasets to optimizing data flows, you will be at the forefront of innovation in automotive technology.

• Data Pipeline Development: Design and maintain robust data pipelines, ensuring seamless data flow from various sources to storage and analytical platforms.
• Data Modeling: Craft and optimize data models for efficient data retrieval and analysis, balancing performance and scalability.
• Data Quality Assurance: Implement rigorous testing and validation protocols to guarantee the accuracy and integrity of data.
• Troubleshooting and System Optimization: Proactively identify and solve data processing issues, enhancing the performance and reliability of data systems.
• Big Data Processing: Engage in large-scale data projects, utilizing frameworks like Apache Spark for efficient data handling and processing.
• Real-Time Data Streaming: Work on cutting-edge projects that process and analyze data in real-time, ensuring swift data delivery and insights.

Whatever you’re working on, here’s the lowdown on the technical skills you’re likely to have:

• At least 3 years of experiencein data engineering.
• Experience working with public cloud platforms, especially AWS.
• Experience with IaC tools, such as Terraform.
• Ability to code in at least one programming language (Python, Java, JavaScript, Scala) is a must.
• Experience working with and creating data-driven architectures, preferably in public cloud platforms.
• Experience using a querying language such as SQL.
• Experience with Big Data and data modelling.
• Advanced knowledge of multiple SQL/noSQL data environments and a strong. understanding of why and when to use a particular solution.
• Familiarity with tools in the Big Data ecosystem (like Hadoop, Cassandra, Apache Hive, Apache Spark, and others alike).
• Experience with orchestration tools like Airflow is a big plus.

CTW's Daily Toolkit

Skills

• Embrace agile practices, especially Scrum, leveraging its events to foster a high-performing team.
• Be cross-functional, developing new skills to enable a team that owns the entire product lifecycle.
• Apply standard engineering practices using tools like version control, IDEs, and dashboards for organized development.
• Adopt DevOps best practices, including Continuous Integration/Continuous Deployment (CI/CD) pipelines for building, testing, and deploying reliable software.
• Operate software post-deployment, including outside working hours if needed (extra compensation applies).
• Travel abroad occasionally to support all phases of product development.
• Have excellent English oral and written skills. German communication skills will be a plus.
• We value diversity and encourage applications from individuals with varied backgrounds and experiences. For this position, candidates must already have the legal right to work in Portugal at the time of application.

Who We Are

• Growth and Innovation: Be a key player in shaping the future of mobility through innovative ideas.
• Engineering Excellence: Immerse yourself in an environment valuing engineering process, where we create, innovate, lead, and make a tangible impact.
• Autonomy and Collaboration: Thrive in a culture that blends autonomy with teamwork, contributing to groundbreaking developments.
• Continuous Learning: Embrace constant skill enhancement in a dynamic workplace, where your contributions impact cutting-edge technology projects.

Join Critical TechWorks on our journey of innovation, excellence, and redefining the future of motion, one product at a time.

At Critical Techworks we are creating an inclusive culture that values our people's different viewpoints and experiences, regardless of sexual orientation, gender identity, age, color, ethnicity, religion, or disability.

Our recruitment process is flexible and accessible to ensure fairness. If you need any adjustments to showcase your skills, let us know during your application. We’ll support you with care and confidentiality.

As part of our recruitment process, we may use AI-supported tools to assist in screening and organizing applications. These tools support our recruiters in structuring information, but do not make hiring decisions independently. All recruitment decisions are made by qualified human recruiters with full oversight.

Critical TechWorks is a joint venture to lead the future of motion. Established in 2018, Critical TechWorks is a company formed as a result of a partnership between BMW Group and Critical Software. Critical TechWorks is exclusively put together to support BMW in building software for its future driving machines.