Lead Data Engineer

Ministry of Justice; City of Westminster; On-site; GBP 59,000 - 71,000; Full time

Benefits offered by this job

Pension contributions

Flexible working arrangements

Job summary

Ministry of Justice is seeking a Lead Data Engineer to head a team of data engineers and deliver automated data flows connecting operations to analytics platforms. You will promote best practices, guide architecture decisions, and work with AWS services to deploy scalable solutions in a secure, compliant environment.

The role requires strong leadership, deep data engineering expertise, and the ability to influence across technical and non-technical stakeholders.

Qualifications

• Strong proficiency in Python and SQL for data engineering tasks.
• Experience with AWS services to deploy solutions at scale.
• Ability to lead and mentor team members, shaping technical direction.
• Understanding of SDLC, testing, version control and CI/CD practices.

Responsibilities

• Lead a team of Data Engineers to build robust, automated data flows.
• Develop analytical pipelines and self-service data tools for analytics and BI.
• Collaborate with stakeholders to understand needs and define project goals.
• Maintain technical documentation and governance of data assets.
• Mentor junior colleagues and drive continuous improvement across the team.
• Support Analytics Platform users in utilizing data and tools.

Skills

Python

SQL

AWS

Team leadership

Tools

Git

GitHub Actions

pytest

CI/CD

Job description

You will work in a team of Data Engineers, alongside Analytics Engineers, to bridge the gap between data producers and data users, developing analytical pipelines and self-service tools to acquire and transform data, making it available on our Analytical Platform. A summary of the type of work our teams are involved in can be found in our handbook. This role aligns against Lead Data Engineer from the Government Digital and Data Framework., As a Lead Data Engineer your work might involve:

• leading a team of Data Engineers to build robust and scalable automated data flows to connect operational systems, data for analytics and business intelligence (BI) systems
• creating and promoting best practices within the team and wider
• applying knowledge of systems integration to your work
• championing data engineering across the department and wider
• collaborating with a range of stakeholders to understand their needs and establish project goals and deliverables
• using a range of AWS services to deploy your solutions in a reproducible, sustainable manner

You may also be expected to carry out the following day to day activities:

• updating and maintaining technical documentation
• leading and organising team building and development initiatives for data engineering and wider
• supporting the Ministry of Justice\'s Analytical Platform users to use our data and tools
• line managing more junior colleagues
• keeping up to date with current trends and best practices in engineering for continuous improvement

At interview stage, you will be assessed against the following Success Profile elements - Experience, Technical and the following Behaviours: Changing and Improving, Managing a Quality Service

Appointments are made strictly in merit order. In the event that two or more candidates receive identical interview scores, strong technical proficiency in one or more programming languages, preferably including Python and SQL will be applied as the primary lead criterion to determine the final merit order. A reserve list may be held for up to 12 months, from which further appointments may be made.

Note: This content includes policies and information related to Artificial Intelligence usage, terms and conditions, flexible working arrangements, diversity and inclusion, and security clearances. See the official guidance for full details.

Technical

• Strong technical proficiency in one or more programming languages, preferably including Python and SQL
• Understanding of the software development lifecycle, including testing (e.g. pytest), version control (e.g. git) and CI/CD (e.g. GitHub Actions)
• Designing and building data pipelines that bring complex data sources together into useful and usable data sets
• Communicate with technical and non-technical stakeholders, and manage their expectations
• Leading and setting the technical direction for a team and developing the capability of others

Protected salary and benefits information from the original posting is preserved in the sections below as required:

• £58,511 - £65,329 (National) to £63,343 - £70,725 (London)
• Other salary ranges and allowances may apply depending on location (National/London)

Base salary and benefits are described in the original posting and should be consulted in full for precise figures and eligibility. This includes details on pension contributions, annual leave, flexible working arrangements, and other employee benefits.