Senior Azure Databricks Data Platform Lead (Unity Catalog)

Microsoft

Atherstone

Hybrid

GBP 47,000 - 53,000

Full time

32 hours ago

Be an early applicant

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Benefits offered by this job

Hybrid work model

12-month contract

Job summary

Allegis Global Solutions is seeking a senior data engineering contractor to modernise an existing Azure data platform for Microsoft. You will lead Unity Catalog migration, redesign data orchestration, and reduce cloud spend while guiding architecture, reviews, and delivery.

The role demands 8–15+ years in data engineering, strong Azure Databricks experience, and a track record of production-grade data platforms. Hybrid, 12-month contract with on-site three days a week in Atherstone.

Qualifications

Azure Databricks (expert level)
Azure Data Factory (ADF)
SQL
Python
Apache Spark
Cloud platform optimisation and cost reduction
Data platform migrations or modernization programmes
Git and modern code review practices
CI/CD implementation
Monitoring, observability, and incident management

Responsibilities

Lead the technical delivery of a data platform modernization programme.
Own the design and execution of Unity Catalog migration across key data domains.
Define and implement modern orchestration patterns, replacing legacy workflows.
Improve performance and reduce costs across Databricks clusters, Spark jobs, storage, and scheduling.
Establish engineering standards for CI/CD, testing, monitoring/observability, data quality and production readiness.
Review architecture decisions and critical pull requests.
Mentor engineers and provide technical guidance.
Work closely with platform, product, analytics, and business stakeholders.
Deliver migrations with minimal business disruption.