Data Engineering Maestro: Real-Time & Big Data

Critical TechWork; Braga; Presencial; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

Critical TechWorks in Portugal seeks a Data Engineer to design and optimize data pipelines, model data architectures, and ensure data quality across large-scale automotive data platforms.

You will work with AWS, Terraform, Spark, SQL/noSQL environments, and apply DevOps and agile practices in a fast-paced, cross-functional team. Experience with real-time data streaming and big data tooling is a plus. A minimum of three years in data engineering and legal right to work in Portugal are required.

Qualificações

• At least 3 years of experience in data engineering.
• Experience with public cloud platforms, especially AWS.
• Experience with IaC tools, such as Terraform.
• Ability to code in Python, Java, JavaScript, or Scala.
• Experience working with and creating data-driven architectures, preferably in public cloud platforms.
• Experience using SQL/noSQL querying languages.
• Experience with Big Data and data modeling.
• Advanced knowledge of multiple SQL/NoSQL data environments and diverse Big Data tools.
• Familiarity with Big Data ecosystem tools such as Hadoop, Cassandra, Hive, Spark, etc.
• Experience with orchestration tools like Airflow is a plus.

Responsabilidades

• Data Pipeline Development: Design and maintain robust data pipelines.
• Data Modeling: Craft and optimize data models for retrieval and analysis.
• Data Quality Assurance: Implement testing and validation of data.
• Troubleshooting and System Optimization: Resolve data processing issues and improve performance.
• Big Data Processing: Handle large-scale projects using Spark and related tools.
• Real-Time Data Streaming: Work on real-time data processing projects.

Conhecimentos

Data engineering

Programming languages

Data modeling

SQL / NoSQL querying

Big Data concepts

DevOps practices

Cloud-native data architectures

Ferramentas

AWS

Terraform

Apache Spark

Airflow

Hadoop

Cassandra

Apache Hive

Descrição da oferta de emprego

Critical TechWorks in Portugal seeks a Data Engineer to design and optimize data pipelines, model data architectures, and ensure data quality across large-scale automotive data platforms.

You will work with AWS, Terraform, Spark, SQL/noSQL environments, and apply DevOps and agile practices in a fast-paced, cross-functional team. Experience with real-time data streaming and big data tooling is a plus. A minimum of three years in data engineering and legal right to work in Portugal are required.