Lead Data Engineer

The Money and Pensions Service

United States

Remote

USD 119,000 - 158,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Annual leave 30 days
Pension scheme
Cycle to work
Life assurance
Volunteer days

Job summary

The Money and Pensions Service (MaPS) is seeking a Lead Data Engineer to drive the architecture, development and operation of our data platform. This hands-on leadership role focuses on scalable data products, platform standards and secure delivery.

You will mentor Data Engineers, translate business requirements into practical architectural decisions, and guide a multi-workstream delivery within MaPS' data ecosystem.

Qualifications

  • Strong experience designing, building and operating Microsoft cloud data platforms, including Microsoft Fabric and Synapse.
  • Experience translating objectives into roadmaps and communicating trade-offs to stakeholders.
  • Experience leading Data Engineers and fostering a collaborative engineering culture.

Responsibilities

  • Lead data platform architecture and detailed technical design.
  • Translate strategic goals into implementation options and a platform roadmap.
  • Own, document and maintain data engineering standards and reusable patterns.
  • Develop platform technology and shared capabilities; evaluate new features.
  • Lead technical delivery across multiple workstreams and mentor engineers.
  • Design for operational ownership with DataOps, monitoring and runbooks.

Skills

Azure Synapse
Python PySpark
SQL expertise
Data platform design
CI/CD
Data governance
Cloud security

Tools

Azure Synapse
Azure Data Factory
Microsoft Fabric
CI/CD tooling

Job description

Join us at the Money and Pensions Service (MaPS) as a Lead Data Engineer. The Lead Data Engineer provides technical leadership for the architecture, development and operation of the MaPS data platform. Working with the Principal Data Engineer, you will translate strategic platform goals into practical architectural decisions, engineering approaches and a prioritised technical roadmap, enabling secure, reliable and reusable data products across MaPS. You will lead the technical evolution of the platform, including migration from Azure Synapse to Microsoft Fabric and support for expanding data needs. You will own data engineering technical standards and solution patterns, ensuring that platform development balances delivery needs with security, maintainability, performance and cost. As part of a larger data engineering team, you will manage, lead and mentor Data Engineers, building capability and shared ownership of engineering quality. This is a hands-on leadership role: you will guide technical delivery, resolve complex engineering problems and develop reusable platform capabilities rather than focusing solely on individual project solutions. You will work with architecture, governance, security, DataOps, analysts and delivery stakeholders to turn business requirements into sustainable data solutions. You will challenge constructively, explain technical choices and trade-offs clearly, and ensure that architectural decisions and engineering practices support MaPS services and the people who use them. The role



Responsibilities


  • Lead data platform architecture and detailed technical design, working within agreed enterprise architecture and governance.

  • Translate strategic goals into implementation options, technical decisions, delivery sequencing and a platform development roadmap agreed with the Principal Data Engineer.

  • Own, document and maintain data engineering technical standards, reference architectures and reusable patterns, including data layering, modelling, naming, ingestion, transformation, testing and deployment. Embed their use through design reviews, code reviews and technical assurance.

  • Develop the platform’s technology and shared capabilities. Evaluate new features through proportionate investigation and proofs of concept, recommending adoption based on business value, security, supportability, performance and cost rather than technology alone.

  • Lead technical delivery across multiple workstreams, resolving cross-project dependencies and ensuring solutions contribute to the shared platform rather than creating unnecessary duplication.

  • Line manage allocated Data Engineers, setting clear objectives, supporting development and wellbeing, and managing workload and delivery expectations. Coach and mentor engineers across the wider team, delegate meaningful technical ownership and build shared capability.

  • Remain hands-on in designing and developing complex pipelines, integrations and data models. Translate requirements into reusable, tested, production-ready solutions and provide technical direction and constructive review for other engineers.

  • Work with data owners, suppliers, analysts and governance and security colleagues to define data contracts, quality controls, access models, metadata, lineage and retention requirements. Implement appropriate engineering controls for secure and ethical data use and sharing, applying open standards and open-by-default principles where appropriate.

  • Embed disciplined engineering practices, including source control, automated testing, CI/CD, release management and environment configuration. Maintain clear architectural decisions, technical documentation and reusable implementation guidance.

  • Design for operational ownership with DataOps, including monitoring, structured logging, alerting, recovery and support runbooks. Lead resolution of complex technical issues, address root causes and manage technical debt, reliability, capacity and cost improvements.



Requirements


  • Strong experience designing, building and operating Microsoft cloud data platforms, including practical Microsoft Fabric experience and an understanding of Azure Synapse and Azure Data Factory in migration and integration scenarios.

  • Experience taking responsibility for data platform architecture and technical design, translating strategic objectives into implementation roadmaps and explaining decisions and trade-offs to technical and non-technical stakeholders.

  • Experience defining and embedding engineering standards, reusable frameworks and solution patterns across multiple engineers and delivery workstreams.

  • Experience managing, leading and mentoring Data Engineers, including setting objectives, supporting professional development, delegating responsibility and building a collaborative engineering culture.

  • Advanced SQL and strong Python/PySpark skills, with practical experience of Spark, lakehouse and warehouse architectures, layered data processing and analytical data modelling.

  • Experience delivering robust ingestion, integration and transformation pipelines using APIs, files and databases, including incremental processing, schema change, error handling and automated data quality validation.

  • Experience of platform migration or significant modernisation, including dependency assessment, reconciliation, testing and controlled transition to production.

  • Practical understanding of cloud identity and access management, data protection, governance, metadata, lineage, retention and secure data sharing, including how to implement these through engineering controls.

  • Experience of source control, code review, automated testing and CI/CD, with clear technical documentation and effective agile delivery practices.

  • Experience designing for production support, including observability, incident investigation, recovery and operational handover, and improving platform reliability, performance and cost.

  • Ability to work with business stakeholders, architects, governance, security and operational teams to clarify requirements, challenge assumptions and agree sustainable solutions.

  • Ability to evaluate emerging platform capabilities, assess risks and benefits, and make proportionate, evidence-based technology recommendations.

  • Ability to coordinate technical delivery across a larger team, balancing immediate delivery commitments with platform development, technical debt and longer-term goals.



The Money and Pensions Service (MaPS) is based in Bedford, in a recently renovated modern office. This is a wonderful opportunity for you to become an integral part of a dynamic organisation, working to help people across the UK.



Values


  • Caring

  • Connecting

  • Transforming



Inclusive Working Environment

By fostering our values, we are immensely proud of the inclusive working environment that we have created. The diversity of our people is a strength that we embrace and wish to build upon, so we are committed to attracting people of all backgrounds. We work hard to ensure that we have a progressive approach to inclusion, equity, and belonging. We really do want our colleagues to \"bring their whole selves to work.\" Our colleague and ally networks encompass LGBTQ+, neurodiversity, women's health, men's health, ethnicity, and diversity.



Benefits


  • Generous Annual Leave - 30 days plus Bank Holidays

  • Pension scheme - contributions matched 2 to 1 (employer contribution up to 10% of your salary)

  • Interest-free loan for season tickets for buses and trains

  • Cycle to work Scheme

  • Subsidised eye tests & flu jabs

  • Life assurance scheme

  • Give as you earn scheme

  • Employee assistance programme (EAP)

  • PAM Assist and PAM Life scheme (Wellbeing)

  • Enhanced family and sick pay

  • Paid volunteering (2 days a year)

  • Recognition Scheme

  • Discounts portal to numerous retailers

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Architect - Data & Analytics Platforms
Lead Data Architect - Data & Analytics Platforms

Capita plc • United States

Remote
USD 150,000 - 210,000
Homebased working
Pension/retirement benefits
Learning Academy access
+1
Senior Data Engineer
Senior Data Engineer

Nucleus Financial • United States

Remote
USD 130,000 - 180,000
non-contributory pension
bonus
enhanced parental leave
+3
Senior Data Engineer
Senior Data Engineer

Government Recruitment Service • Uxbridge (MA)

On-site
USD 55,000 - 67,000
Pension contributions
Annual leave
Flexible working
Senior Data Engineer
Senior Data Engineer

RVU • United States

Remote
USD 120,000 - 160,000
25 days holiday + bank holidays (up to
Pension via People's Pension
Private medical insurance (subsidised)
+4
Engineering Manager - PaaS Cloud
Engineering Manager - PaaS Cloud

Sainsbury’s Group • United States

Hybrid
USD 119,000 - 172,000
Colleague discounts and benefits
Pensions scheme
Life cover
+10
Client Support Administrator (Part-time)
Client Support Administrator (Part-time)

Duncan & Toplis Group • Stamford (CT)

Hybrid
USD 22,000 - 30,000
Hybrid working
CPD and training
Flexible hours
Lead Data Engineer at NPAworldwide Birmingham, AL
Lead Data Engineer at NPAworldwide Birmingham, AL

NPAworldwide • Birmingham (AL)

Hybrid
USD 67,000 - 94,000
Flexible hours
Hybrid/remote work
Learning budget
+5
Pensions Data Service Delivery Manager
Pensions Data Service Delivery Manager

Capita • United States

Remote
USD 86,000 - 119,000
Holiday entitlement
Pension plan
Cycle2work
+7
Engineering Manager
Engineering Manager

IMP Software Ltd. • United States

Remote
USD 91,000 - 123,000
27 days holiday + bank holidays
Enhanced maternity/paternity leave
AXA Health insurance (dental, optical,
+4
Advanced Insight Analyst
Advanced Insight Analyst

Amplius • Milton-Freewater (OR)

Hybrid
USD 62,000 - 83,000
Flexible working
Pension contributions
Cycle to work scheme
+4