Senior Data Platform Engineer: Azure Airflow & Lakehouse Lead

Capitole Consulting; Barcelona; On-site; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Professional development allowance (€1

Private health insurance

Meal and transport vouchers

Capitole Club discounts

Wellhub subscription

Team events and sports clubs

Job summary

Capitole Consulting is seeking a Senior Data Platform Engineer to evolve a large Azure-based data platform and lakehouse architecture. You will own foundations used by data product teams, including Airflow orchestration, Terraform-based infrastructure, and Unity Catalog governance, while staying hands-on and close to the consuming teams.

The role focuses on platform reliability, cost efficiency, and observability, with leadership in architectural reviews and engineering standards.

Qualifications

• 6+ years of data engineering experience focusing on platform/infrastructure topics.
• Strong Python and SQL (advanced) with PySpark and Delta Lake semantics.
• Hands-on production experience with Apache Airflow.
• Hands-on Azure resource management, including IaC with Terraform.
• Experience with Databricks and Unity Catalog governance.
• Familiarity with cost management and governance of cloud resources.

Responsibilities

• Own the Airflow platform: environments, deployment, upgrades, configuration and DAG standards.
• Migrate from Azure Data Factory to Airflow and evolve the orchestration framework.
• Ensure Airflow is reliable and observable with monitoring and alerts.
• Manage Azure resources with Terraform: Databricks, Data Lake Storage, Key Vault, networking.
• Handle secrets/identity and access management (Key Vault, Azure AD, Unity Catalog).
• Oversee Azure DevOps pipelines, multi-environment promotion and Terraform state.
• Contribute to Data Catalog/Quality framework and overall observability across the platform.

Skills

Python

SQL

Data engineering

Cross-functional collaboration

Problem solving

Tools

Apache Airflow

Terraform

Databricks

Unity Catalog

Azure DevOps

Key Vault

Azure RBAC

Job description

Capitole Consulting is seeking a Senior Data Platform Engineer to evolve a large Azure-based data platform and lakehouse architecture. You will own foundations used by data product teams, including Airflow orchestration, Terraform-based infrastructure, and Unity Catalog governance, while staying hands-on and close to the consuming teams.

The role focuses on platform reliability, cost efficiency, and observability, with leadership in architectural reviews and engineering standards.