Hybrid Data Engineer (Lisbon/Porto) – Build Scalable Data Platforms

Affinity; Lisboa; Híbrido; EUR 48 000 - 72 000; Tempo integral

Resumo da oferta

Affinity, a Portuguese technology consulting company with 350+ associates and offices in Lisbon, Porto and Leiria, invites a data engineer to join our data platform team. You will design, implement, and optimize scalable data workflows and platforms across cloud and on-premise environments.

You will work with Spark, Kafka, and SQL across Azure and AWS, collaborate with cross-functional teams, ensure data quality and security, and help shape analytics for European projects.

Qualificações

• Experience with cloud platforms and Azure for data engineering.
• Proficiency in Python, Java or Scala.
• Strong knowledge of Spark, Kafka and other Big Data tools.
• Data modeling and database design principles.
• Familiarity with ETL tools and distributed data processing.
• SQL proficiency with relational and NoSQL databases.
• Experience with Git and cloud data services.

Responsabilidades

• Design, build, and maintain scalable data platforms.
• Collect, process, and analyze large, complex data sets from diverse sources.
• Develop data workflows using Spark and Apache Beam.
• Collaborate with cross-functional teams to ensure data accuracy and integrity.
• Ensure data security and privacy through proper access controls.
• Extract data from various sources including databases, files, and APIs.
• Monitor performance and optimize for high availability and scalability.

Conhecimentos

Cloud platforms

Python

Java

Scala

SQL

Distributed systems

Data modeling

Git

Cloud data engineering

Ferramentas

Spark

Kafka

Hadoop

Hive

Flink

ElasticSearch

Sqoop

Flume

Impala

Kafka Streams

Connect

Druid

AWS S3

Azure Data Factory

Descrição da oferta de emprego

Affinity, a Portuguese technology consulting company with 350+ associates and offices in Lisbon, Porto and Leiria, invites a data engineer to join our data platform team. You will design, implement, and optimize scalable data workflows and platforms across cloud and on-premise environments.

You will work with Spark, Kafka, and SQL across Azure and AWS, collaborate with cross-functional teams, ensure data quality and security, and help shape analytics for European projects.