Senior data Engineer 2 (contract)

Microsoft Corporation; Atherstone; Hybrid; GBP 47,000 - 53,000; Full time

Job summary

Microsoft Corporation in Atherstone, England is seeking a senior data engineering contractor to modernize an existing Azure data platform. You will lead a Unity Catalog migration, redesign data orchestration, and drive platform reliability while actively participating in architecture reviews, code reviews, and delivery.

The role requires hands-on leadership with substantial Azure Databricks experience, strong SQL and Python skills, and a track record of reducing cloud costs.

Qualifications

• Must lead production-grade Azure data platforms.
• Experience delivering large migration or transformation projects.
• Ability improve performance, reliability and cloud costs.

Responsibilities

• Lead the technical delivery of a data platform modernization programme.
• Own the design and execution of Unity Catalog migration across key data domains.
• Define and implement modern orchestration patterns, replacing legacy workflows.
• Improve performance and reduce costs across Databricks clusters, Spark jobs, storage and scheduling.
• CI/CD and testing responsibilities.
• Monitoring, observability and incident management.
• Production readiness and review of architecture decisions.
• Mentor engineers and provide technical guidance.
• Collaborate with platform, analytics and business stakeholders.
• Deliver migrations with minimal business disruption.

Skills

SQL

Python

Cloud platform optimisation

Data platform migrations

Git and code review

CI/CD implementation

Monitoring and observability

Tools

Azure Databricks

Job description

Atherstone, England, United Kingdom (Hybrid)

Contract (1 year)

Published 10 days ago

Technical Leadership & DevOps

Data Platform Modernization & Migration

Please note that this is a contract role providing services to Microsoft through external staffing partners of Allegis Global Solutions (AGS). If you are selected for this role, you will be employed by AGS and will not be an employee of Microsoft.

Role Summary

This is a hands‑on senior data engineering contractor role focused on modernising an existing Azure data platform. The person will lead a Unity Catalog migration, redesign data orchestration processes, improve platform reliability, and reduce cloud spend. They will provide technical leadership while remaining heavily involved in architecture, code reviews, and delivery. This is best suited to someone who has already led Azure Databricks transformation projects in production environments.

Key Responsibilities

• Lead the technical delivery of a data platform modernization programme.
• Own the design and execution of Unity Catalog migration across key data domains.
• Define and implement modern orchestration patterns, replacing legacy workflows.
• Improve performance and reduce costs across Databricks clusters, Spark jobs, storage, and scheduling.
• CI/CD
• Testing
• Monitoring and observability
• Production readiness
• Review architecture decisions and critical pull requests.
• Mentor engineers and provide technical guidance.
• Work closely with platform, product, analytics, and business stakeholders.
• Deliver migrations with minimal business disruption.

Must-Have Skills (Non-Negotiable)

Technical

• SQL
• Python
• Cloud platform optimisation and cost reduction
• Data platform migrations or modernization programmes
• Git and modern code review practices
• CI/CD implementation
• Monitoring, observability, and incident management

Experience

• Led delivery of production‑grade Azure data platforms.
• Successfully delivered large migration or transformation projects.
• Experience making measurable improvements in performance, reliability, and cloud costs.
• Comfortable operating as a technical lead rather than only an individual contributor.

Nice-to-Have Skills

• Unity Catalog migration experience
• Gaming industry experience
• Power BI
• Tableau
• Lakehouse architecture expertise
• Data reliability engineering (DRE) background

Experience Level Guidance

• 8-15+ years in Data Engineering / Data Platform Engineering
• 3-5+ years working heavily with Azure Databricks
• Previous Technical Lead, Lead Data Engineer, Principal Data Engineer, or Data Platform Architect experience

Location: Atherstone, England

Duration: 12 months

Pay Range: £255/day to £287/day (PAYE)

Weekly Schedule: 40 hours

As a workplace, Allegis Global Solutions focuses on relationships – with each other, our clients and our candidates. In fact, serving others is one of our core values. We support open communication and recognize that giving constructive criticism can be even harder than receiving it. We appreciate the fearless and the passionate, who force us to be better. Everything we do sits on a pillar of diversity, equity and inclusion - diverse perspectives, backgrounds and ideas achieve innovation and make us successful.

Please note that we may use artificial intelligence (AI) tools to screen, assess, or select applicants for this position. These tools may analyze application materials and assist our team in identifying candidates whose qualifications best match the requirements of the role. If you have questions about our use of AI in the hiring process, or would like more information, please contact us.