Data Engineer

NHS Connecting for Health; Greater London; Hybrid; GBP 42,000 - 45,000; Full time

Job summary

A public archival institution seeks a skilled Data Engineer to enhance the management of archival data. You will design robust data pipelines, automate processes, and work with diverse data sources to support public access to the UK’s documentary heritage. The position offers generous benefits including a competitive salary and opportunities for professional growth.

Benefits

Generous benefits package

On-site gym

Training and development opportunities

Annual leave entitlement of 22 days

Qualifications

• Hands-on experience with AWS including identity and access management.
• Proficiency in Infrastructure as Code (Terraform) and scripting.
• Ability to communicate effectively across technical and non-technical teams.

Responsibilities

• Design and develop scalable data pipelines (ETL/ELT).
• Automate data cleansing, validation, and enrichment.
• Collaborate with product teams and archivists to deliver data-centric solutions.

Skills

AWS

Python

Infrastructure as Code (Terraform)

Scripting (Bash, PowerShell)

Data handling

Tools

Talend

SQL

NoSQL

XML

JSON

Job description

Overview

The National Archives is an inclusive, entrepreneurial, and forward-thinking place to work. We are a 200-year-old institution with a broad mission to preserve and provide access to the UK’s documentary heritage. Our strategic vision, Archives for Everyone, aims to become a 21st century national archive — a different kind of cultural and heritage institution. This role contributes to that journey.

Job Overview

Salary: £42,002 plus an additional market supplement £2,998 per annum

Contract type: permanent

Band: E / HEO

Closing date: Monday 20 October at midnight

Shape the Future of the Nation’s Memory

As a Data Engineer in our Digital Archiving team, you will help transform how archival data is managed, structured, and enriched, unlocking its value for public use and discovery.

Why Join Us?

• Your work will directly support the preservation and accessibility of the UK’s documentary heritage, including a new Parliamentary Archives project.
• We nurture talent, value every voice, and support personal and professional development. You’ll join a collaborative, forward-looking team where your ideas are heard and your growth is championed.
• Work with a modern tech stack including Python, SQL, Shell scripting, Talend, XSLT, XML, CSV, JSON, RDF, RDBMS, NoSQL, and graph databases. You’ll help design and build scalable data pipelines, automate data processing, and develop tools for data enrichment and integration.
• We invest in your learning, offering opportunities to develop your skills in emerging technologies and data practices that enhance public access to archives.

What You’ll Do

• Design and develop scalable, repeatable data pipelines (ETL/ELT) to process, transform, and load archival data.
• Automate data cleansing, validation, standardisation, and enrichment.
• Develop and document schemas, ontologies, and models for archival data.
• Integrate data from diverse sources and formats, supporting better discovery and reuse.
• Collaborate with product teams, archivists, and developers to deliver practical, data-centric solutions.
• Contribute to a culture of openness, quality, and continuous improvement.

Role Eligibility and Requirements

Two full-time positions are available. Requests for part-time working, flexible working and job share will be considered, taking account of operational needs. A combination of onsite and home working is available; applicants should be able to regularly travel to our Kew site for a minimum of 60% of work time.

Application Process

• Interviews will be held on-site during the week commencing 3 November and will include a technical test. At interview, you will be assessed on essential criteria listed below.
• Application: you will be asked to upload your CV and answer questions, including examples of AWS work and experience with infrastructure as code.

Essential Criteria

• Hands-on experience with AWS (including identity and access management).
• Proficiency in Infrastructure as Code (Terraform) and scripting (Python, Bash, or PowerShell).
• Ability to communicate effectively across technical and non-technical teams.

Desirable Criteria

• Awareness of archival data value and ethical responsibilities.
• Understanding of probabilistic, messy, or uncertain data handling.
• Experience in Agile development environments.
• Familiarity with archival principles or metadata standards (e.g. EAD, Dublin Core).
• Experience in semantic web technologies and entity extraction tools.

The Civil Service is committed to attracting, retaining and investing in talent. See Civil Service People Plan and Diversity & Inclusion (D&I) Strategy for more information.

Benefits

Generous benefits package, including pension, sports and social club facilities, on-site gym, discounts at the on-site cafe, and opportunities for training and development. Annual leave entitlement of 22 days per year (rising to 25 after the first year, and up to 30 after six years) plus 10.5 public and privilege holidays per year.

Note: Moving to The National Archives may affect eligibility for childcare vouchers; other government schemes may apply (e.g., Tax-Free Childcare).

Reasonable Adjustments

If you need adjustments to participate in the recruitment process, please contact [e-mail]. We will provide adjustments as required.

Security and Nationality

Successful candidates must pass a disclosure and barring security check. Nationality requirements apply as specified by the Civil Service. See the vacancy details for full information.

Working for the Civil Service

The Civil Service Code sets standards of behaviour for civil servants. We recruit on merit and are committed to equal opportunities, including Disability Confident and Redeployment schemes. This vacancy is part of the Great Place to Work for Veterans initiative.

Contact Point for Applicants

• The National Archives Recruitment Team, [e-mail]