Data Engineer - Azure

Affinity; Lisboa; Híbrido; EUR 48 000 - 72 000; Tempo integral

Resumo da oferta

Affinity, a Portuguese technology consulting company with 350+ associates and offices in Lisbon, Porto and Leiria, invites a data engineer to join our data platform team. You will design, implement, and optimize scalable data workflows and platforms across cloud and on-premise environments.

You will work with Spark, Kafka, and SQL across Azure and AWS, collaborate with cross-functional teams, ensure data quality and security, and help shape analytics for European projects.

Qualificações

• Experience with cloud platforms and Azure for data engineering.
• Proficiency in Python, Java or Scala.
• Strong knowledge of Spark, Kafka and other Big Data tools.
• Data modeling and database design principles.
• Familiarity with ETL tools and distributed data processing.
• SQL proficiency with relational and NoSQL databases.
• Experience with Git and cloud data services.

Responsabilidades

• Design, build, and maintain scalable data platforms.
• Collect, process, and analyze large, complex data sets from diverse sources.
• Develop data workflows using Spark and Apache Beam.
• Collaborate with cross-functional teams to ensure data accuracy and integrity.
• Ensure data security and privacy through proper access controls.
• Extract data from various sources including databases, files, and APIs.
• Monitor performance and optimize for high availability and scalability.

Conhecimentos

Cloud platforms

Python

Java

Scala

SQL

Distributed systems

Data modeling

Git

Cloud data engineering

Ferramentas

Spark

Kafka

Hadoop

Hive

Flink

ElasticSearch

Sqoop

Flume

Impala

Kafka Streams

Connect

Druid

AWS S3

Azure Data Factory

Descrição da oferta de emprego

We are a Portuguese technology consulting company with a strong outward look to the rest of Europe. We have 14 years of experience in the market and reached the status of “large company” in 2018. We now have more than 350 associates and we keep growing, with offices in Lisbon, Porto, and Leiria, as well as projects in more than 20 countries. Our company is part of the national top 10, with a growth of 250% in the last years.

Work-life balance is part of our DNA. We focus on win-win-win relationships, defining every moment of interaction as an experience with unique conditions. Working with Affinity is a “Lifetime Experience” where you will have the opportunity to demonstrate all your potential and to be successful on every level.

#welcometoyourfut"ure

What you need to have/know:

• Experience with cloud platforms and services for data engineering (Azure);
• Proficiency in programming languages like Python, Java, or Scala;
• Use of Big Data Tools such as Spark, Flink, Kafka, Elastic Search, Hadoop, Hive, Sqoop, Flume, Impala, Kafka Streams and Connect, Druid, etc.;
• Knowledge of data modeling and database design principles;
• Familiarity with data integration and ETL tools (e.g., Apache Kafka, Talend);
• Understanding of distributed systems and data processing architectures;
• Strong SQL skills and experience with relational and NoSQL databases;
• Familiarity with cloud platforms and services for data engineering (e.g., AWS S3, Azure Data Factory);
• Experience with version control tools such as Git.

Work Setup

• Hybrid Mode - Lisbon or Porto

What we expect from you:

• Design, build, and maintain scalable data platforms;
• Collect, process, and analyze large and complex data sets from various sources;
• Develop and implement data processing workflows using data processing framework technologies such as Spark, and Apache Beam;
• Collaborate with cross-functional teams to ensure data accuracy and integrity;
• Ensure data security and privacy through proper implementation of access controls and data encryption;
• Extraction of data from various sources, including databases, file systems, and APIs;
• Monitor system performance and optimize for high availability and scalability.

What will encourage you

• Being part of a company that has a disruptive impact on the technology market;
• Being part of a technology-focused company built around relationships, simplicity, and efficiency;
• Access to a career path directly proportional to your performance and interests;
• The real possibility of becoming part of our internal team.
• A collaborative environment where ownership, autonomy, and continuous development are valued.

Ready to take ownership of your next challenge?

Join Affinity and start your #LifetimeExperience.