Senior Azure Data Engineer (Databricks) - Hybrid

korecv; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Discretionary bonus

Health insurance

Pension

Mental health support

Five weeks annual leave

Job summary

Capco is seeking a Senior Azure Data Engineer (Databricks) in London to design, build, and deploy scalable data pipelines on Azure using Databricks. You will collaborate with clients to translate data needs into production-ready solutions and contribute to engineering best practices across cross-functional teams.

The role emphasizes responsible AI, secure data handling, and end-to-end data lifecycle management in a hybrid work environment within a leading consulting culture.

Qualifications

• Proven hands-on experience with the Databricks platform and Azure data engineering at scale.
• Strong Python, PySpark and SQL skills for distributed data processing.
• Experience delivering end-to-end data pipelines in modern Data Lakehouse architectures.

Responsibilities

• Design and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services.
• Build real-time, event-driven solutions using Kafka and Azure Event Hubs with AI-enabled approaches to improve delivery.
• Develop and maintain CI/CD pipelines using Azure DevOps, Jenkins, and GitHub Actions for reliable deployments.
• Collaborate with clients and multidisciplinary teams to translate business requirements into scalable, production-ready data solutions.
• Promote clean code, data lifecycle optimization, and responsible AI tools to improve collaboration and delivery.

Skills

Databricks

Python

PySpark

SQL

Spark

Azure

Kafka

Azure Event Hubs

Delta Lake

Unity Catalog

CI/CD

Azure DevOps

Jenkins

GitHub Actions

Scala

Java

Tools

Azure DevOps

Jenkins

GitHub Actions

Job description

Capco is seeking a Senior Azure Data Engineer (Databricks) in London to design, build, and deploy scalable data pipelines on Azure using Databricks. You will collaborate with clients to translate data needs into production-ready solutions and contribute to engineering best practices across cross-functional teams.

The role emphasizes responsible AI, secure data handling, and end-to-end data lifecycle management in a hybrid work environment within a leading consulting culture.