Senior Data Engineer AWS

Hexa People; Lisboa; Presencial; EUR 40 000 - 60 000; Tempo integral

Resumo da oferta

A leading technology center is looking for Data Engineers to join their new Data Engineering team in Lisbon. The successful candidates will implement cloud-based data pipelines, manage complex datasets, and collaborate with cross-functional teams. Mandatory skills include experience with AWS technologies, strong SQL abilities, and proficiency in Python. This role promises a dynamic environment with opportunities for professional growth in a company recognized for its excellence.

Qualificações

• Proven experience in data pipelining and related technologies on AWS.
• Good skills in writing complex SQL queries, knowledge of DBT core is a plus.
• Strong understanding of data platform architecture and governance principles.

Responsabilidades

• Implement cloud-based data pipelines to transform raw data into structured formats.
• Architect and deploy high-performance data pipelines.
• Assemble and manage large, complex datasets for business requirements.

Conhecimentos

Cloud-native data pipeline development

Python

SQL

Data governance principles

Formação académica

University degree in Engineering or related discipline

Ferramentas

AWS (S3, ECS, Lambda, RDS, SNS/SQS, IAM, Cloudwatch)

Snowflake

Orchestration tools (e.g., Step Function, Airflow)

Descrição da oferta de emprego

Driven by innovation and excellence, client has been shaping the future since 2016.

As the technology center (Founded in Geneva), we are proud to support one of Europes leading independent wealth and asset managers by developing cutting-edge solutions for its business lines.

Recognized as a Great Place to Work for eight years, we first established ourselves in Luxembourg before expanding to Lisbon in 2022. Today, we are entering a new phase with the kick-off of our Data Engineering team in Lisbon. Our ambition is clear: to grow a strong team of experts and build a thriving community of specialists across locations.

We are looking for passionate individuals eager to be part of this new chapter. You will join a dynamic and collaborative environment, contribute to meaningful projects, and help shape the future of Data Engineering, always with a long-term vision and a deep commitment to the Groups success

We are seeking 5 Data Engineers with expertise in cloud-native data pipeline development within AWS environments and Snowflake, medior, senior and lead profiles. The ideal candidate will have a strong background in data architecture, data modelling, and the ability to implement high-performance data solutions that support our business objectives. You will join the Scrum team of one of our product as part of the development team and collaborate with data analysts and internal teams to ensure the delivery and the run of the product in line with the sprint backlog.

In this context, your role will include:

• Implement cloud-based data pipelines to transform raw data from various sources (e.g. APIs, FTP, SFTP) into structured formats.
• Architect and deploy high-performance (batch and streaming) data pipelines with monitoring and alerting system implemented for data analysts and business users.

Data Management

• Assemble and manage large, complex datasets that meet both functional and non-functional business requirements.
• Support data governance initiatives by cataloguing and describing metadata.
• Implement functional and non-functional tests on data.

Data Modelling and Analysis

• Perform data modelling and design appropriate database objects (views, tables, procedures, scripts).
• Analyze and optimize queries for performance and scalability.

Collaboration and Support Production

• Participate in troubleshooting, monitoring, and user support activities.
• Work closely with cross-functional teams to gather requirements and implement solutions.
• Code according to established architecture guidelines and focus on improving the scalability and performance of data products.
• Stay updated with the latest industry trends and technologies to enhance data engineering practices.

Our ideal candidates should have:

Mandatory skills

• University degree in Engineering or related discipline
• Proven experience in data pipelining and related technologies on AWS (S3, ECS, Lambda, RDS, SNS/SQS, IAM, Cloudwatch)
• Extensive experience with analytical database programming (SQL) and unstructured data/graph databases
• Good skills in writing complex SQL queries, knowledge of DBT core is a plus
• Proficient in Python and data-related libraries (e.g. Pandas, Boto3)
• Strong understanding of data platform architecture and data governance principles
• Deep knowledge of developing on Snowflake is a plus
• Adoption of CI/CD practices
• Experience using orchestration tools (e.g. Step Function, Airflow, Prefect)
• Experience with version control tool such as GitHub or GitLab
• Knowledge of data visualization tools (e.g., Tableau) is a plus
• Knowledge of Infrastructure as code (e.g. CloudFormation, Terraform)
• AWS certification (e.g., Solution Architect or equivalent) is preferred
• Snowflake certification (e.g. SnowPro Core) is preferred