Lead Data Engineer

Money & Pensions Service

England

Hybrid

GBP 73,000 - 90,000

Full time

43 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

30 days annual leave + bank holidays
Pension scheme matching
Cycle to work scheme
Eye tests & flu jabs
Employee assistance program

Job summary

Money and Pensions Service (MaPS) in Bedford is seeking a Lead Data Engineer to provide technical leadership for our data platform. You will translate strategic goals into architectural decisions, guide the migration to Microsoft Fabric, and own data engineering standards while mentoring a team of Data Engineers.

This hands-on leadership role requires designing scalable data pipelines, ensuring security, performance and cost efficiency, and collaborating with governance, DataOps and delivery

Qualifications

  • Experience designing, building and operating Microsoft cloud data platforms.
  • Translate strategic objectives into implementation roadmaps and decisions.
  • Embed engineering standards, reusable frameworks and patterns across multiple engineers.
  • Mentor Data Engineers and build collaborative engineering culture.
  • Advanced SQL and Python/PySpark with lakehouse/warehouse architectures.

Responsibilities

  • Lead data platform architecture and technical design within enterprise governance.
  • Translate goals into platform development roadmap with the Principal Data Engineer.
  • Maintain data engineering standards, reference architectures and reusable patterns.
  • Lead multi-workstream delivery, manage cross-project dependencies.
  • Hands-on design of complex pipelines, data models and production-ready solutions.
  • Coach engineers and ensure secure, compliant data practices.

Skills

Cloud data platforms
Azure Synapse
Microsoft Fabric
Azure Data Factory
SQL
Python / PySpark
Data modelling
Leadership

Tools

Spark
Git
CI/CD
DataOps

Job description

Lead Data Engineer
£81,500 per annum
Bedford
Permanent
Hybrid working

Join us at the Money and Pensions Service (MaPS) as a Lead Data Engineer. The Lead Data Engineer provides technical leadership for the architecture, development and operation of the MaPS data platform. Working with the Principal Data Engineer, you will translate strategic platform goals into practical architectural decisions, engineering approaches and a prioritised technical roadmap, enabling secure, reliable and reusable data products across MaPS.

You will lead the technical evolution of the platform, including migration from Azure Synapse to Microsoft Fabric and support for expanding data needs. You will own data engineering technical standards and solution patterns, ensuring that platform development balances delivery needs with security, maintainability, performance and cost.

As part of a larger data engineering team, you will manage, lead and mentor Data Engineers, building capability and shared ownership of engineering quality. This is a hands‑on leadership role: you will guide technical delivery, resolve complex engineering problems and develop reusable platform capabilities rather than focusing solely on individual project solutions.

You will work with architecture, governance, security, DataOps, analysts and delivery stakeholders to turn business requirements into sustainable data solutions. You will challenge constructively, explain technical choices and trade‑offs clearly, and ensure that architectural decisions and engineering practices support MaPS services and the people who use them.

Key Responsibilities:
  • Lead data platform architecture and detailed technical design, working within agreed enterprise architecture and governance.
  • Translate strategic goals into implementation options, technical decisions, delivery sequencing and a platform development roadmap agreed with the Principal Data Engineer.
  • Own, document and maintain data engineering technical standards, reference architectures and reusable patterns, including data layering, modelling, naming, ingestion, transformation, testing and deployment. Embed their use through design reviews, code reviews and technical assurance.
  • Develop the platform’s technology and shared capabilities. Evaluate new features through proportionate investigation and proofs of concept, recommending adoption based on business value, security, supportability, performance and cost rather than technology alone.
  • Lead technical delivery across multiple workstreams, resolving cross‑project dependencies and ensuring solutions contribute to the shared platform rather than creating unnecessary duplication.
  • Line manage allocated Data Engineers, setting clear objectives, supporting development and wellbeing, and managing workload and delivery expectations. Coach and mentor engineers across the wider team, delegate meaningful technical ownership and build shared capability.
  • Remain hands‑on in designing and developing complex pipelines, integrations and data models. Translate requirements into reusable, tested, production‑ready solutions and provide technical direction and constructive review for other engineers.
  • Work with data owners, suppliers, analysts and governance and security colleagues to define data contracts, quality controls, access models, metadata, lineage and retention requirements. Implement appropriate engineering controls for secure and ethical data use and sharing, applying open standards and open‑by‑default principles where appropriate.
  • Embed disciplined engineering practices, including source control, automated testing, CI/CD, release management and environment configuration. Maintain clear architectural decisions, technical documentation and reusable implementation guidance.
  • Design for operational ownership with DataOps, including monitoring, structured logging, alerting, recovery and support runbooks. Lead resolution of complex technical issues, address root causes and manage technical debt, reliability, capacity and cost improvements.
Essential Skills and Experience
Essential:
  • Strong experience designing, building and operating Microsoft cloud data platforms, including practical Microsoft Fabric experience and an understanding of Azure Synapse and Azure Data Factory in migration and integration scenarios.
  • Experience taking responsibility for data platform architecture and technical design, translating strategic objectives into implementation roadmaps and explaining decisions and trade‑offs to technical and non‑technical stakeholders.
  • Experience defining and embedding engineering standards, reusable frameworks and solution patterns across multiple engineers and delivery workstreams.
  • Experience managing, leading and mentoring Data Engineers, including setting objectives, supporting professional development, delegating responsibility and building a collaborative engineering culture.
  • Advanced SQL and strong Python/PySpark skills, with practical experience of Spark, lakehouse and warehouse architectures, layered data processing and analytical data modelling.
  • Experience delivering robust ingestion, integration and transformation pipelines using APIs, files and databases, including incremental processing, schema change, error handling and automated data quality validation.
  • Experience of platform migration or significant modernisation, including dependency assessment, reconciliation, testing and controlled transition to production.
  • Practical understanding of cloud identity and access management, data protection, governance, metadata, lineage, retention and secure data sharing, including how to implement these through engineering controls.
  • Experience of source control, code review, automated testing and CI/CD, with clear technical documentation and effective agile delivery practices.
  • Experience designing for production support, including observability, incident investigation, recovery and operational handover, and improving platform reliability, performance and cost.
  • Ability to work with business stakeholders, architects, governance, security and operational teams to clarify requirements, challenge assumptions and agree sustainable solutions.
  • Ability to evaluate emerging platform capabilities, assess risks and benefits, and make proportionate, evidence‑based technology recommendations.
  • Ability to coordinate technical delivery across a larger team, balancing immediate delivery commitments with platform development, technical debt and longer‑term goals.
About Us

The Money and Pensions Service (MaPS) is based in Bedford, in a recently renovated modern office. This is a wonderful opportunity for you to become an integral part of a dynamic organisation, working to help people across the UK.

At the heart of the Money and Pensions Service are our values – caring, connecting, and transforming, which are the foundation of our success. They permeate every area of our work and define all our business relationships and the way we work with each other. We’re not only looking for the best people to come and work for us, but we need people who align themselves with our values:

  • Caring
    We care about our colleagues and the people whose lives we are here to transform.
  • Connecting
    We will transform lives through our ability to make positive connections.
  • Transforming
    We are committed to transforming lives and making a positive societal impact.
Our Inclusive Working Environment

By fostering our values, we are immensely proud of the inclusive working environment that we have created. The diversity of our people is a strength that we embrace and wish to build upon, so we are committed to attracting people of all backgrounds. We work hard to ensure that we have a progressive approach to inclusion, equity, and belonging. We really do want our colleagues to “bring their whole selves to work.”

Our colleague and ally networks encompass LGBTQ+, neurodiversity, women’s health, men’s health, ethnicity, and diversity.

What We Offer
  • Generous Annual Leave – 30 days plus Bank Holidays
  • Pension scheme – contributions matched 2 to 1 (employer contribution up to 10% of your salary)
  • Interest‑free loan for season tickets for buses and trains
  • Cycle to work Scheme
  • Subsidised eye tests & flu jabs
  • Life assurance scheme
  • Give as you earn scheme
  • Employee assistance programme (EAP)
  • PAM Assist and PAM Life scheme (Wellbeing)
  • Enhanced family and sick pay
  • Paid volunteering (2 days a year)
  • Recognition Scheme
  • Discounts portal to numerous retailers
Flexible Working

At MaPS, we take pride in our flexible approach to work. Hybrid working is a voluntary, non‑contractual arrangement and our headquarters in Bedford will be your contractual place of work. The number of days that anyone will be able to work at home will be determined primarily by business needs, but personal and other relevant circumstances will also be considered. If you are successful, any opportunities for hybrid working, including whether a hybrid working arrangement is suitable for you, will be discussed with you prior to you taking up your post.

Recruitment Process:
  1. Initial Review:Our recruiter will review your CV. If it looks like a good fit, they'll reach out to arrange a call.
  2. Hiring Manager Review:If the recruiter thinks you're suitable, your CV will be passed to the hiring manager, who will decide whether to invite you for an interview.
  3. Interview:If selected, you'll participate in a virtual interview. This single-stage process allows you to demonstrate your skills and experience through competency-based questions.
Career Development

In MaPS, we take career development seriously. We actively encourage and support applications from our existing MaPS colleagues. However, we do follow the Civil Service Commissioner recruitment principles, which means that you will be required to participate in a full, open, and fair process.

Reserve List

If you are successful at interview, we operate a reserve list where your details will be held for up to 6 months. Should a vacancy come available in that time with the same essential criteria, reserve list candidates will be offered that position with no further assessment required.

Application Process

The law requires that selection for appointment to the Civil Service is on merit on the basis of fair and open competition as outlined in the Civil Service Commission’s Recruitment Principles. If you feel your application has not been treated in accordance with the Recruitment Principles, and you wish to make a complaint, in the first instance, you should contact the Money and Pensions Service via email: recruitment@maps.org.uk. If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission: Visit the Civil Service Commission website.

Job Reference: MaPS01405
Close Date: 1/12/2026

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Technical Architect
Principal Technical Architect

Money and Pensions Service • Bedford

Hybrid
GBP 90,000 - 110,000
Generous Annual Leave – 30 days plus
Pension scheme – employer matched up
Cycle to work scheme
+1
Lead Delivery Manager
Lead Delivery Manager

Money and Pensions Service • Bedford

Hybrid
GBP 72,000 - 88,000
Annual leave 30 days
Pension matched 2:1
Cycle to work
+3
Post Connections Operations Lead
Post Connections Operations Lead

Reed Talent Solutions • Bedford

Hybrid
GBP 77,000 - 94,000
Generous Annual Leave
Pension scheme
Cycle to work
+4
Test Engineer
Test Engineer

Money & Pensions Service • Cambridgeshire and Peterborough

Hybrid
GBP 49,000 - 59,000
Generous annual leave – 30 days + bank
Pension scheme with employer match
Interest-free season ticket loan
+3
Lead D365 Power Platform Developer
Lead D365 Power Platform Developer

Money and Pensions Service • Bedford

Hybrid
GBP 74,000 - 100,000
Annual leave 30 days
Pension scheme
Cycle to work
+2
Lead D365 / Power Platform Developer (P-4182927684)
Lead D365 / Power Platform Developer (P-4182927684)

Reed Talent Solutions • Bedford

Hybrid
GBP 90,000 - 130,000
Pension scheme
Cycle to work
Eye tests & flu jabs
+4
Transition Implementation Lead
Transition Implementation Lead

Money and Pensions Service • Bedford

Hybrid
GBP 68,000 - 83,000
30 days annual leave
Pension (2:1 matched up to 10%)
Season ticket loan
+8
Lead Product Manager
Lead Product Manager

Reed Talent Solutions • Bedford

Hybrid
GBP 78,000 - 96,000
Pension scheme
Cycle to work scheme
Eye tests & flu jabs
+2
Support Hub Analyst
Support Hub Analyst

Money and Pensions Service • Cambridgeshire and Peterborough

Hybrid
GBP 36,000 - 44,000
30 days’ annual leave plus bank hours
Pension (2:1 matched contribution up 2
Season ticket loan
+8
Support Hub Lead
Support Hub Lead

Money and Pensions Service • Cambridgeshire and Peterborough

Hybrid
GBP 53,000 - 64,000
Pension 2:1 up to 10%
Season ticket loan
Cycle to work
+7