Lead Data Engineer — Azure & Databricks Lakehouse

TP; Lisboa; On-site; EUR 70,000 - 90,000; Full time

Job summary

TP is hiring a Senior Data Engineer to design and optimise data pipelines on an Azure and Databricks lakehouse. You will own data ingestion, modelling and delivery across the business, collaborating with data scientists, BI developers and stakeholders.

You will build scalable ETL/ELT pipelines using Azure Data Factory, Databricks and Spark, manage Delta Lake tables, and drive data governance, quality and security. Strong SQL, Python and cloud skills are essential.

Qualifications

• 5+ years in data engineering or related field.
• Hands-on experience with Azure cloud services.
• Experience with Databricks, Spark, and Delta Lake.
• Advanced SQL and Python, with distributed data processing.
• CI/CD and IaC using Terraform, GitHub Actions, or Azure DevOps.
• Strong data modelling fundamentals.
• Excellent English communication for diverse audiences.
• Ownership and autonomy in distributed teams.
• Nice to have: Kafka, Azure certifications, MLOps, Purview/Unity Catalog, GDPR knowledge.

Responsibilities

• Design and build scalable ETL/ELT data pipelines with Azure Data Factory, Databricks and Apache Spark.
• Build and manage data lakes and Delta tables on Azure Data Lake Storage Gen2 and Delta Lake.
• Optimise Spark workloads for performance, scalability and cost efficiency.
• Embed data quality, data governance and security practices across pipelines.
• Partner with analytics teams to enable self-service reporting in Power BI and Power Apps.
• Monitor and troubleshoot production jobs using Azure Monitor and Databricks job runs.
• Mentor junior data engineers and set standards for coding, architecture and DevOps.

Skills

Data engineering

Azure cloud

Databricks

Spark Py/Scala

SQL

Python

CI/CD

IaC

Terraform

GitHub Actions

Azure DevOps

Data modelling

English comms

Education

Bachelor's degree in Computer Science or related

Tools

Azure Data Factory

Databricks

Apache Spark

Delta Lake

Azure Data Lake Storage Gen2

Power BI

Power Apps

Azure Monitor

Terraform

GitHub Actions

Azure DevOps

Job description

TP is hiring a Senior Data Engineer to design and optimise data pipelines on an Azure and Databricks lakehouse. You will own data ingestion, modelling and delivery across the business, collaborating with data scientists, BI developers and stakeholders.

You will build scalable ETL/ELT pipelines using Azure Data Factory, Databricks and Spark, manage Delta Lake tables, and drive data governance, quality and security. Strong SQL, Python and cloud skills are essential.