Data Engineer

Adecco Recruitment; Lisboa; Híbrido; EUR 60.000 - 100.000; Jornada completa

Descripción de la vacante

Adecco Recruitment in Lisbon is seeking a Databricks-focused data engineer for a global technology consultant. You will build and optimize Databricks pipelines, apply Medallion Architecture, and collaborate with data scientists and architects on governance and security.

This full-time role offers a flexible hybrid work arrangement in Lisbon, working with Azure Data Factory, Airflow, and CI/CD practices to deliver scalable, reliable data products for financial services.

Formación

• Experience with Databricks as the primary engineering platform.
• Solid understanding of Medallion Architecture and lakehouse principles.
• Proficiency in PySpark and distributed processing.
• Experience with Azure Data Factory, Airflow and workflow orchestration.
• Familiarity with Azure DevOps CI/CD, Git workflows and automated deployments.
• Comfortable working in Agile, collaborative and multicultural environments.

Responsabilidades

• Build, optimize and maintain end-to-end Databricks pipelines for performance, reliability and scalability.
• Apply Medallion Architecture to design structured, maintainable data layers.
• Develop distributed processing solutions with PySpark.
• Design and orchestrate workflows using ADF and Airflow to improve automation and observability.
• Contribute to Databricks Lakehouse standards, tuning and cluster optimization.
• Integrate CI/CD practices using Azure DevOps with automated testing and versioning.
• Collaborate with data scientists, architects and business stakeholders to operationalize data products for finance.
• Support governance, security and data documentation across the Databricks ecosystem.

Conocimientos

Databricks

Medallion Architecture

PySpark

Azure Data Factory

Airflow

Azure DevOps CI/CD

Git workflows

Agile

Descripción del empleo

Join a team where Databricks is at the heart of the engineering strategy and where your work powers large‑scale financial platforms used across global markets.

Location: Flexible Hybrid in Lisbon

Contract: Full‑time, direct contract with a global technology consulting leader.

Our client is a world‑renowned technology consulting company partnering with a major multinational in banking and financial services. This long‑term program is part of a data and cloud modernization journey centred on Databricks, scalable architectures and high‑quality data products. You will join an international engineering environment dedicated to transforming complex data into reliable, governed and business‑critical assets.

Responsibilities

• Build, optimize and maintain end-to-end Databricks pipelines, ensuring performance, reliability and scalability.
• Apply Medallion Architecture principles to design structured, maintainable and production‑ready data layers.
• Develop distributed processing solutions with PySpark, aligned with best engineering practices.
• Design and orchestrate workflows using ADF and Airflow, improving automation and observability.
• Contribute to the evolution of Databricks Lakehouse standards, performance tuning and cluster optimization.
• Integrate CI/CD practices using Azure DevOps, with automated testing and version control.
• Collaborate closely with data scientists, architects and business stakeholders to operationalize data products and ensure alignment with financial‑sector requirements.
• Support governance, security and documentation across the Databricks ecosystem.

Required Skills

• Strong, demonstrable experience with Databricks as the primary engineering platform.
• Solid understanding of Medallion Architecture and modern lakehouse principles.
• Proficiency in PySpark and distributed processing frameworks.
• Experience with Azure Data Factory, Airflow and workflow orchestration.
• Familiarity with Azure DevOps CI/CD, Git workflows and automated deployments.
• Comfortable working in Agile, collaborative and multicultural environments.

Nice to Have

• Advanced experience with Azure DevOps for pipeline automation.
• Background in data testing or test framework development.
• Exposure to investment banking or financial services data landscapes.