Senior Data Engineer (Azure & Databricks)

TP; Lisboa; Presencial; EUR 70.000 - 90.000; Jornada completa

Descripción de la vacante

TP is hiring a Senior Data Engineer to design and optimise data pipelines on an Azure and Databricks lakehouse. You will own data ingestion, modelling and delivery across the business, collaborating with data scientists, BI developers and stakeholders.

You will build scalable ETL/ELT pipelines using Azure Data Factory, Databricks and Spark, manage Delta Lake tables, and drive data governance, quality and security. Strong SQL, Python and cloud skills are essential.

Formación

• 5+ years in data engineering or related field.
• Hands-on experience with Azure cloud services.
• Experience with Databricks, Spark, and Delta Lake.
• Advanced SQL and Python, with distributed data processing.
• CI/CD and IaC using Terraform, GitHub Actions, or Azure DevOps.
• Strong data modelling fundamentals.
• Excellent English communication for diverse audiences.
• Ownership and autonomy in distributed teams.
• Nice to have: Kafka, Azure certifications, MLOps, Purview/Unity Catalog, GDPR knowledge.

Responsabilidades

• Design and build scalable ETL/ELT data pipelines with Azure Data Factory, Databricks and Apache Spark.
• Build and manage data lakes and Delta tables on Azure Data Lake Storage Gen2 and Delta Lake.
• Optimise Spark workloads for performance, scalability and cost efficiency.
• Embed data quality, data governance and security practices across pipelines.
• Partner with analytics teams to enable self-service reporting in Power BI and Power Apps.
• Monitor and troubleshoot production jobs using Azure Monitor and Databricks job runs.
• Mentor junior data engineers and set standards for coding, architecture and DevOps.

Conocimientos

Data engineering

Azure cloud

Databricks

Spark Py/Scala

SQL

Python

CI/CD

IaC

Terraform

GitHub Actions

Azure DevOps

Data modelling

English comms

Educación

Bachelor's degree in Computer Science or related

Herramientas

Azure Data Factory

Databricks

Apache Spark

Delta Lake

Azure Data Lake Storage Gen2

Power BI

Power Apps

Azure Monitor

Terraform

GitHub Actions

Azure DevOps

Descripción del empleo

We're hiring a Senior Data Engineer to lead the design and optimisation of data pipelines on a modern Azure and Databricks lakehouse platform. You'll own how data is ingested, modelled and delivered across the business — working shoulder to shoulder with data scientists, BI developers and business stakeholders.

What you'll do

• Design and build scalable ETL/ELT data pipelines with Azure Data Factory, Databricks and Apache Spark
• Build and manage data lakes and Delta tables on Azure Data Lake Storage Gen2 and Delta Lake
• Optimise Spark workloads for performance, scalability and cost efficiency
• Embed data quality, data governance and security practices across every pipeline;
• Partner with analytics teams to enable self-service reporting in Power BI and Power Apps
• Monitor and troubleshoot production jobs using Azure Monitor and Databricks job runs
• Mentor junior data engineers and set standards for coding, architecture and DevOps

What you'll bring

• 5+ years in data engineering or a closely related field
• Strong hands-on expertise across Azure cloud services
• Proven experience with Databricks, Spark (PySpark or Scala) and Delta Lake
• Advanced SQL and Python, plus distributed data processing
• CI/CD and Infrastructure as Code — Terraform, GitHub Actions or Azure DevOps
• Solid data modelling fundamentals
• Excellent communication in English, with technical and non-technical audiences alike
• Ownership, autonomy and comfort working in a distributed, global team
• Nice to have: Experience with Kafka or Azure Event Hubs, Azure certifications, MLOps, data governance tools (Purview, Unity Catalog), and knowledge of GDPR or data security practices.