Data Engineer - Azure

Affinity; Setúbal; Híbrido; EUR 50 000 - 75 000; Tempo integral

Resumo da oferta

Affinity, a Portuguese technology consulting company, is seeking a Data Engineer to design, build, and maintain scalable data platforms. You will collect and process large datasets, implement data workflows with Spark and Apache Beam, and collaborate across teams to ensure data integrity and security.

Hybrid work setup in Lisbon or Porto, with exposure to leading Big Data technologies, cloud platforms, and modern ETL practices.

Qualificações

• Experience with cloud platforms and services for data engineering (Azure)
• Proficiency in programming languages like Python, Java, or Scala
• Use of Big Data tools such as Spark, Flink, Kafka, ElasticSearch, Hadoop, Hive, Sqoop, Flume, Impala, Kafka Streams and Connect, Druid
• Knowledge of data modeling and database design principles
• Familiarity with data integration and ETL tools (Apache Kafka, Talend)
• Understanding of distributed systems and data processing architectures
• Strong SQL skills and experience with relational and NoSQL databases
• Familiarity with cloud platforms for data engineering (AWS S3, Azure Data Factory)
• Experience with version control tools such as Git

Responsabilidades

• Design, build, and maintain scalable data platforms
• Collect, process, and analyze large and complex data sets from various sources
• Develop and implement data processing workflows using Spark and Apache Beam
• Collaborate with cross-functional teams to ensure data accuracy and integrity
• Ensure data security and privacy through proper access controls and data encryption
• Extraction of data from various sources, including databases, file systems, and APIs
• Monitor system performance and optimize for high availability and scalability

Conhecimentos

Python

Java

Scala

SQL

Ferramentas

Spark

Kafka

Hadoop

Hive

Talend

Git

Flink

Beam

Druid

Descrição da oferta de emprego

We are a Portuguese technology consulting company with a strong outward look to the rest of Europe. We have 14 years of experience in the market and reached the status of "large company" in 2018. We now have more than 350 associates and we keep growing, with offices in Lisbon, Porto, and Leiria, as well as projects in more than 20 countries. Our company is part of the national top 10, with a growth of 250% in the last years.

Work-life balance is part of our DNA. We focus on win-win-win relationships, defining every moment of interaction as an experience with unique conditions. Working with Affinity is a "Lifetime Experience" where you will have the opportunity to demonstrate all your potential and to be successful on every level.

#welcometoyourfuture

What you need to have/know:

• Experience with cloud platforms and services for data engineering (Azure);
• Proficiency in programming languages like Python, Java, or Scala;
• Use of Big Data Toolssuch as Spark, Flink, Kafka, Elastic Search, Hadoop, Hive, Sqoop, Flume, Impala, Kafka Streams and Connect, Druid, etc.;
• Knowledge of data modeling and database design principles;
• Familiarity with dataintegration and ETL tools (e.G., Apache Kafka, Talend);
• Understanding of distributed systems and data processing architectures;
• Strong SQL skills andexperience with relational and NoSQL databases;
• Familiarity with cloud platforms and services for data engineering (e.G., AWS S3, Azure Data Factory);
• Experience with version control tools such as Git.

Work Setup

• Hybrid Mode - Lisbon or Porto

What we expect from you:

• Design, build, and maintain scalable data platforms;
• Collect, process, andanalyze large and complex data sets from various sources;
• Develop and implementdata processing workflows using data processing framework technologies such as Spark, and Apache Beam;
• Collaborate with cross-functional teams to ensure data accuracy and integrity;
• Ensure data security and privacy through proper implementation of access controls and data encryption;
• Extraction of data from various sources, including databases, file systems, and APIs;
• Monitor system performance and optimize for high availability and scalability.

What will encourage you

• Being part of a company that has a disruptive impact on the technology market;
• Being part of a technology-focused company built around relationships, simplicity, and efficiency;
• Access to a career path directly proportional to your performance and interests;
• The real possibility of becoming part of our internal team.
• A collaborative environment where ownership, autonomy, and continuous development are valued.

Ready to take ownership of your next challenge?

Join Affinity and start your #LifetimeExperience.