Senior Azure Data Engineer (Databricks) – Real-Time Pipelines

Capco

Glasgow

Hybrid

GBP 90,000 - 120,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Discretionary bonus
Pension
Health insurance
Life insurance
Critical illness cover
CareFirst

Job summary

Capco is seeking a Senior Azure Data Engineer (Databricks) in the UK within a hybrid model across Scotland. You will design, build, and deploy scalable data pipelines on the Databricks platform on Azure, partnering with clients to translate data needs into practical solutions.

This role sits in Capco's Scotland office, with potential Edinburgh or Glasgow client-based work. You will contribute to best practices and continuous improvement within cross-functional teams.

Qualifications

  • Proven hands-on experience with Databricks on Azure and orchestration.
  • Strong Python, PySpark, and SQL skills for distributed data pipelines.
  • Experience across ingestion, transformation, and serving layers in data lakehouse architectures.

Responsibilities

  • Design and develop robust pipelines using DeltaLake and Spark Structured Streaming.
  • Build real-time event-driven solutions with Kafka and Azure Event Hubs.
  • Apply DevOps principles to CI/CD pipelines using Azure DevOps, Jenkins, or GitHub Actions.
  • Collaborate with clients to translate data needs into strategic technical solutions.
  • Champion clean code and data lifecycle optimization.

Skills

Databricks
Python
SQL
PySpark
DevOps
GDPR compliance

Tools

Azure
DeltaLake
Spark
Unity Catalog

Job description

Capco is seeking a Senior Azure Data Engineer (Databricks) in the UK within a hybrid model across Scotland. You will design, build, and deploy scalable data pipelines on the Databricks platform on Azure, partnering with clients to translate data needs into practical solutions.

This role sits in Capco's Scotland office, with potential Edinburgh or Glasgow client-based work. You will contribute to best practices and continuous improvement within cross-functional teams.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Azure Data Engineer (Databricks) - Real-Time Pipelines
Senior Azure Data Engineer (Databricks) - Real-Time Pipelines

Capco • City of Edinburgh

Hybrid
GBP 70,000 - 100,000
Discretionary bonus
Health insurance
Pension
+3
Senior Azure Data Engineer (Databricks)
Senior Azure Data Engineer (Databricks)

Capco • City of Edinburgh

Hybrid
GBP 70,000 - 100,000
Discretionary bonus
Health insurance
Pension
+3
Senior Azure Data Engineer (Databricks)
Senior Azure Data Engineer (Databricks)

Capco • Glasgow

Hybrid
GBP 90,000 - 120,000
Discretionary bonus
Pension
Health insurance
+3
Azure Databricks Lead Engineer | Data & Platform Ops
Azure Databricks Lead Engineer | Data & Platform Ops

Sogeti • United Kingdom

Hybrid
GBP 70,000 - 110,000
Azure Databricks Engineer for Data Pipelines & Lakes
Azure Databricks Engineer for Data Pipelines & Lakes

Capgemini • Greater London

Hybrid
GBP 70,000 - 100,000
Disability Confident Employer (Level 2
Senior Azure Data Engineer — Databricks, ADF, Python
Senior Azure Data Engineer — Databricks, ADF, Python

CGI Group Inc. • City of Westminster

Hybrid
GBP 65,000 - 90,000
Insurance coverage
Medical benefits
Pension plan
+3
Azure Databricks Engineer — Hybrid Data Solutions
Azure Databricks Engineer — Hybrid Data Solutions

Capgemini • Greater London

Hybrid
GBP 70,000 - 110,000
Databricks Data Engineer: Azure Data Pipelines & AI-Ready
Databricks Data Engineer: Azure Data Pipelines & AI-Ready

Dufrain Consulting Ltd • England

On-site
GBP 70,000 - 100,000
Senior Databricks Data Engineer — Big Data Pipelines
Senior Databricks Data Engineer — Big Data Pipelines

Korn Ferry Group • Greater London

Remote
GBP 83,025 - 101,475
Data Engineer: Azure Databricks & Cloud Pipelines (Hybrid)
Data Engineer: Azure Databricks & Cloud Pipelines (Hybrid)

Standard Life plc • City of Edinburgh

Hybrid
GBP 35,000 - 45,000
Bonus
Private medical cover
38 days annual leave
+5