Lead Data Engineer

Serco Group Plc

Manor Park

Hybrid

GBP 120,000 - 150,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Discounts on cinema
Merlin entertainment discounts
Online shopping discounts
Mobile phone plan discounts
Leisure centre memberships

Job summary

Serco Group plc is seeking a Lead Data Engineer to act as the technical authority for data engineering, delivering enterprise-scale data solutions across Azure, AWS, and Databricks.

You will spend ~80% hands-on building data pipelines and 20% leading engineers, shaping data engineering practices and governance across the organisation.

}

Qualifications

  • 8+ years' experience in data engineering within enterprise environments.
  • Advanced proficiency in PySpark and SQL with performance optimization.
  • Experience leading data engineering teams and mentoring engineers.
  • Certifications such as Azure Data Engineer Associate, AWS Data Analytics Specialty, or Databricks Data Engineer are desirable.

Responsibilities

  • Define and enforce data engineering standards, patterns, and best practices across the organisation.
  • Architect and deliver scalable data solutions across Azure, AWS, and Databricks.
  • Lead complex engineering workstreams and guide technical decisions for high-quality delivery.
  • Design and govern enterprise data models, including lakehouse strategies and schema evolution.
  • Champion Databricks as the core data platform and leverage Unity Catalog, Delta Live Tables, and streaming tech.
  • Drive a multi-cloud Databricks strategy for scalable, interoperable platforms across environments.
  • Embed CI/CD for data platforms with automated testing and secure deployments.
  • Implement robust data governance, security, lifecycle controls, and lineage frameworks.

Skills

Databricks
PySpark
SQL
Leadership
Multi-cloud
Stakeholder management

Tools

Azure
AWS
Databricks
Unity Catalog
Delta Lake

Job description

Here at Serco, we're looking for a Lead Data Engineer to act as the technical authority for data engineering, driving architecture, standards, and delivery across a modern multi-cloud ecosystem spanning Azure, AWS, and Databricks.

You'll spend around 80% of your time hands-on, designing and building complex, enterprise-scale data solutions. The remaining 20% is just as critical: leading people, influencing stakeholders, and shaping how data engineering operates across the organisation.

Databricks sits at the heart of our platform - this role requires someone who can deeply leverage the Databricks to enable scalable analytics, data science, and AI use cases across the business.

As Part Of This You'll
  • Define and enforce data engineering standards, patterns, and best practices across the organisation.
  • Architect and deliver scalable, resilient data solutions across Azure, AWS, and Databricks, ensuring enterprise-grade performance and reliability.
  • Lead complex engineering workstreams - unblocking teams, guiding technical decisions, and ensuring high-quality delivery outcomes.
  • Design and govern enterprise data models (conceptual, logical, and physical), including lakehouse zone strategies and schema evolution approaches.
  • Champion Databricks as the core data platform, leveraging lakehouse architecture, Unity Catalog, Delta Live Tables, Structured Streaming, and MLflow.
  • Drive a consistent multi-cloud Databricks strategy, enabling scalable and interoperable data platforms across environments.
  • Optimise performance across pipelines, compute, storage, and orchestration frameworks, embedding cost-efficiency and scalability.
  • Define and implement CI/CD for data platforms, supporting automated testing, secure deployments, and multi-environment promotion.
  • Embed robust data governance, security, and lifecycle controls, ensuring compliance through classification, encryption, access management, and lineage frameworks (e.g., Unity Catalog, Purview, Lake Formation).
  • Act as a senior SME for troubleshooting complex issues, leading root-cause analysis and long-term remediation of data quality and platform challenges.
  • Lead, mentor, and grow a community of Data Engineers, driving technical excellence through coaching, reviews, and knowledge sharing.
  • Operate effectively across a matrix organisation, influencing architecture, cyber, product, and senior stakeholders.
  • Communicate complex technical concepts clearly, articulating trade-offs, risks, and strategic decisions to both technical and non-technical audiences.
  • Expert-level experience in enterprise-scale data engineering, including defining standards, patterns, and overseeing complex delivery.
  • Advanced proficiency in PySpark and SQL, with deep expertise in performance optimisation and scalable data processing.
  • Strong experience in data modelling and large-scale data integration design, with the ability to build robust, maintainable, and testable pipelines.
  • Proven ability to embed testing frameworks, observability, and data quality controls across engineering workflows.
  • Deep hands-on experience with Databricks (essential), including lakehouse architecture, Delta Lake, Unity Catalog, and streaming technologies.
  • Broad cloud expertise across Azure (Synapse, ADF, Event Hub, ADLS Gen2, Purview, Key Vault) and AWS (S3, Glue, EMR, Kinesis, Athena, Lake Formation, IAM), with strong multi-cloud experience.
  • Strong understanding of modern engineering practices, including CI/CD for data platforms, automated deployment pipelines, and environment promotion strategies.
  • Experience designing monitoring, observability, and cost optimisation frameworks, alongside governance, lineage, cataloguing, and compliance at scale.
  • Excellent stakeholder management skills, with the ability to influence senior leaders and work effectively across matrixed organisations.
  • Proven leadership capability, including mentoring, coaching, and developing engineering teams.
  • Strong communication skills, with the ability to translate complex technical concepts into clear business value.

8+ years' experience in data engineering within enterprise environments.

Relevant certifications (desirable), such as Azure Data Engineer Associate, AWS Data Analytics Specialty, or Databricks Data Engineer (Associate/Professional).

Serco discounts which include cinema, merlin entertainment and online shopping discounts, and discounts on mobile phone plans and leisure centre memberships.

A range of benefits to support the health and wellbeing of you and your family such as Employee Assistance Programme, Simply Health Cash Plans, and more.

A wealth of career development training to suit your future aspirations. These range from role specific training, leadership coaching, formal study and much more to support you to build your career with Serco.

A safe and supportive culture.

A Little Bit About Us

At Serco, everyone plays a vital part in delivering essential public services that make a real difference. As a global team of 50,000 people across Defence, Transport, Justice, Immigration, Healthcare, Leisure and Citizen Services, we're united by a shared purpose: to improve the services that communities rely on. Guided by our values of Trust, Care, Innovation and Pride, our people make meaningful impact, every single day.

Our Inclusive Recruitment Commitment

We prioritise your skills and potential. As a Disability Confident Leader, the Top Great British Employer of Veterans 2026, and a Gold Inclusive Employer Standard holder, we champion equal opportunities for all. We offer flexible and hybrid working where possible and make reasonable adjustments throughout recruitment and employment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer
Lead Data Engineer

Serco Group Plc • Hook

Hybrid
GBP 90,000 - 120,000
Hybrid working
Employee discounts
Career development opportunities
+1
Lead Data Engineer
Lead Data Engineer

Serco • Metropolitan Borough of Solihull, Greater London, Hook

Hybrid
GBP 75,000 - 85,000
Company car
5% on target bonus scheme
Private medical insurance
+7
Lead Data Engineer
Lead Data Engineer

Serco Plc • Metropolitan Borough of Solihull, Greater London, Hook

Hybrid
GBP 75,000 - 85,000
Company car
5% on target bonus scheme
Private medical insurance
+2
Lead Data Engineer
Lead Data Engineer

Serco Limited • Metropolitan Borough of Solihull, Greater London, Hook

Hybrid
GBP 75,000 - 85,000
Company car
Bonus scheme
Private medical insurance
+5
Lead Data Engineer
Lead Data Engineer

Serco Group Plc • Metropolitan Borough of Solihull

Hybrid
GBP 75,000 - 85,000
Private medical insurance
Flexible working considered
Pension – 6%
+1
Data & Insights Developer
Data & Insights Developer

Serco Plc • Greater London

Hybrid
GBP 60,000 - 65,000
Company car
Private healthcare
5% on target bonus scheme
+3
Analytics Engineer
Analytics Engineer

Serco • Metropolitan Borough of Solihull

Hybrid
GBP 40,000 - 50,000
Pension 6% employer matched
Annual leave 25 days + bank holidays
Life assurance 2x base salary
+1
Technical Lead
Technical Lead

Serco • Metropolitan Borough of Solihull

Hybrid
GBP 50,000 - 60,000
Pension 6% employer matched
Annual leave 25 days + bank holidays
Life assurance 2x base salary
+1
Senior Data Solutions Architect Data Strategy · Newcastle upon Tyne, UK Remote / Hybrid
Senior Data Solutions Architect Data Strategy · Newcastle upon Tyne, UK Remote / Hybrid

Seriös Group • Newcastle upon Tyne

Hybrid
GBP 60,000 - 80,000
Great salary £60k-£80k
Unlimited holidays
Private healthcare
+4
FM Maintenance Engineer (Mechanical)
FM Maintenance Engineer (Mechanical)

Serco • Uttoxeter

On-site
GBP 36,000 - 44,000
25 days plus bank holidays
Pension scheme (up to 6%)
Internal courses and qualifications (e
+5