Data Engineering Mastermind

Critical TechWork; Braga; Presencial; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

Critical TechWorks in Portugal seeks a Data Engineer to design and optimize data pipelines, model data architectures, and ensure data quality across large-scale automotive data platforms.

You will work with AWS, Terraform, Spark, SQL/noSQL environments, and apply DevOps and agile practices in a fast-paced, cross-functional team. Experience with real-time data streaming and big data tooling is a plus. A minimum of three years in data engineering and legal right to work in Portugal are required.

Qualificações

• At least 3 years of experience in data engineering.
• Experience with public cloud platforms, especially AWS.
• Experience with IaC tools, such as Terraform.
• Ability to code in Python, Java, JavaScript, or Scala.
• Experience working with and creating data-driven architectures, preferably in public cloud platforms.
• Experience using SQL/noSQL querying languages.
• Experience with Big Data and data modeling.
• Advanced knowledge of multiple SQL/NoSQL data environments and diverse Big Data tools.
• Familiarity with Big Data ecosystem tools such as Hadoop, Cassandra, Hive, Spark, etc.
• Experience with orchestration tools like Airflow is a plus.

Responsabilidades

• Data Pipeline Development: Design and maintain robust data pipelines.
• Data Modeling: Craft and optimize data models for retrieval and analysis.
• Data Quality Assurance: Implement testing and validation of data.
• Troubleshooting and System Optimization: Resolve data processing issues and improve performance.
• Big Data Processing: Handle large-scale projects using Spark and related tools.
• Real-Time Data Streaming: Work on real-time data processing projects.

Conhecimentos

Data engineering

Programming languages

Data modeling

SQL / NoSQL querying

Big Data concepts

DevOps practices

Cloud-native data architectures

Ferramentas

AWS

Terraform

Apache Spark

Airflow

Hadoop

Cassandra

Apache Hive

Descrição da oferta de emprego

We’re looking for bright Data Engineer Mastermindsto join our team

At Critical TechWorks, we're not just engineering software; we're sculpting the data landscape of tomorrow. Our Data Engineers are pivotal in transforming the BMW Group into a data-driven powerhouse. From handling vast datasets to optimizing data flows, you will be at the forefront of innovation in automotive technology.

• Data Pipeline Development: Design and maintain robust data pipelines, ensuring seamless data flow from various sources to storage and analytical platforms.

• Data Modeling: Craft and optimize data models for efficient data retrieval and analysis, balancing performance and scalability.

• Data Quality Assurance: Implement rigorous testing and validation protocols to guarantee the accuracy and integrity of data.

• Troubleshooting and System Optimization: Proactively identify and solve data processing issues, enhancing the performance and reliability of data systems.

• Big Data Processing: Engage in large-scale data projects, utilizing frameworks like Apache Spark for efficient data handling and processing.

• Real-Time Data Streaming: Work on cutting-edge projects that process and analyze data in real-time, ensuring swift data delivery and insights.

Whatever you’re working on, here’s the lowdown on the technical skills you’re likely to have:

• At least 3 years of experiencein data engineering.

• Experience working with public cloud platforms, especially AWS.

• Experience with IaC tools, such as Terraform.

• Ability to code in at least one programming language (Python, Java, JavaScript, Scala) is a must.

• Experience working with and creating data-driven architectures, preferably in public cloud platforms.

• Experience using a querying language such as SQL.

• Experience with Big Data and data modelling.

• Advanced knowledge of multiple SQL/noSQL data environments and a strong. understanding of why and when to use a particular solution.

• Familiarity with tools in the Big Data ecosystem (like Hadoop, Cassandra, Apache Hive, Apache Spark, and others alike).

• Experience with orchestration tools like Airflow is a big plus.

CTW's Daily Toolkit

Apart from strong technical skills, our fast-paced, fully agile environment values autonomy, teamwork, interaction, and resilience. Therefore, our PlatformProduct Visionaries should:

• Embrace agile practices, especially Scrum, leveraging its events to foster a high-performing team.

• Be cross-functional, developing new skills to enable a team that owns the entire product lifecycle.

• Apply standard engineering practices using tools like version control, IDEs, and dashboards for organized development.

• Adopt DevOps best practices, including Continuous Integration/Continuous Deployment (CI/CD) pipelines for building, testing, and deploying reliable software.

• Operate software post-deployment, including outside working hours if needed (extra compensation applies).

• Travel abroad occasionally to support all phases of product development.

• Have excellent English oral and written skills. German communication skills will be a plus.

• We value diversity and encourage applications from individuals with varied backgrounds and experiences. For this position, candidates must already have the legal right to work in Portugal at the time of application.

Who we are

Critical TechWorks seeks a distinct and treasured corporate culture, that is very much our own. This means:

• Growth and Innovation: Be a key player in shaping the future of mobility through innovative ideas.

• Engineering Excellence: Immerse yourself in an environment valuing engineering process, where we create, innovate, lead, and make a tangible impact.

• Autonomy and Collaboration: Thrive in a culture that blends autonomy with teamwork, contributing to groundbreaking developments.

• Continuous Learning: Embrace constant skill enhancement in a dynamic workplace, where your contributions impact cutting-edge technology projects.

Join Critical TechWorks on our journey of innovation, excellence, and redefining the future of motion, one product at a time.

At Critical Techworks we are creating an inclusive culture that values our people’s different viewpoints and experiences, regardless of sexual orientation, gender identity, age, color, ethnicity, religion, or disability.

Our recruitment process is flexible and accessible to ensure fairness. If you need any adjustments to showcase your skills, let us know during your application. We’ll support you with care and confidentiality.

As part of our recruitment process, we may use AI-supported tools to assist in screening and organizing applications. These tools support our recruiters in structuring information, but do not make hiring decisions independently. All recruitment decisions are made by qualified human recruiters with full oversight.