Senior Data Engineer

The Alan Turing Institute; Greater London; In loco; GBP 57.000 - 69.000; Tempo pieno

Vantaggi offerti da questo lavoro

Flexible working

Family-friendly policies

Descrizione del lavoro

The Alan Turing Institute in London seeks a Senior Data Engineer to design and operate data pipelines and infrastructure, transforming diverse data into reliable resources for analysts and ML initiatives across cloud and on‑premise environments.

You will shape the data platform, implement data models, serving layers and query interfaces, and collaborate with engineers and partners to meet security and operational constraints while delivering scalable, production‑grade solutions.

Competenze

• Proven experience designing, building and operating production‑grade data pipelines.
• Strong Python engineering skills and experience with modern software practices.
• Experience deploying and managing data infrastructure across cloud and on‑premise environments.

Mansioni

• Design, build and operate robust data pipelines ingesting large varied data sets.
• Develop data models, curated datasets, serving layers and query interfaces for analysts.
• Maintain data platform across cloud and on‑premise with IaC and secure access control.
• Collaborate with engineers and partner teams within security‑constrained environments.
• Establish and enforce data engineering standards, testing, schema and contract management.

Conoscenze

Python

Data pipelines

Cloud platforms

Infrastructure as Code

Strumenti

Infrastructure as Code

Descrizione del lavoro

We are looking for a Senior Data Engineer to design and build the data pipelines and infrastructure that transform complex, heterogeneous data into reliable, well-structured resources. Initially, you will focus on the data layer underpinning analyst-facing tools, including curated datasets, data models, serving layers and query interfaces. As the team's machine learning capabilities grow, this foundation will also support feature pipelines, dataset versioning and evaluation data.

You will establish and maintain the team's data platform across cloud and on-premise environments, covering storage, compute, orchestration and access control. Working within Defence and partner environments, you will design solutions that meet technical and security constraints while enabling effective data use.

This is a senior individual contributor role suited to an experienced data engineer who has built and operated data systems at scale, can set technical direction, and is comfortable working within operationally constrained environments.

HOW YOU WILL MAKE AN IMPACT

• Design, build and operate robust data pipelines that ingest, validate, clean and structure large volumes of varied data, ensuring quality, lineage and reproducibility from the outset.
• Develop the data products that support the team's tools, including data models, curated datasets, serving layers and query interfaces used by analysts and engineers.
• Establish and maintain the team's data platform across cloud and on-premise environments, covering storage, compute, orchestration, access control and infrastructure-as-code.
• Work closely with applied engineers and partner teams to understand how data will be consumed and design solutions that operate effectively within technical and security constraints.
• Set and maintain the team's data engineering standards, including testing, schema and contract management, documentation and secure data handling, supported by effective code review.
• Help shape the team's data architecture and ways of working, mentoring colleagues and advising on the future development of data and platform engineering capabilities.

ABOUT YOU

The successful candidate will be an experienced and pragmatic Senior Data Engineer who is motivated by producing work that has real operational relevance and impact for UK defence and national security. You'll have a proven track record of designing, building and operating production-grade data pipelines across large, diverse and often poorly structured datasets, and be confident working with both batch and streaming data.

With strong Python engineering skills and expertise in modern software development practices, you'll be passionate about developing reliable, scalable systems that are built to last. You'll bring hands‑on experience deploying and managing data infrastructure across cloud and on‑premise environments, using infrastructure‑as‑code to create secure, maintainable platforms. Equally comfortable designing trusted data products as you are shaping data architecture, you'll understand how to create curated datasets, serving layers and data models that downstream users can depend on.

Most importantly, you'll be a collaborative problem‑solver who enjoys working alongside engineers, data scientists, analysts and external partners, balancing technical excellence with the realities of operational delivery in security‑conscious environments.

Please see our portal for a full breakdown of the role.

Closing date for applications: Sunday 08 November 2026 at 23:59 (London, UK BST/GMT)

Terms and Conditions

This position is offered on a fixed‑term basis with a salary of £57,000-£69,000 per annum, plus an excellent benefits package, including flexible working and family‑friendly policies. Employee-only benefits guide | The Alan Turing Institute

Given the close working relationship with government partners, this role requires regular on‑site presence with Defence partners in London 3 days per week, with possible occasional travel to other sites in the UK, in addition to the Institute's standard office attendance expectations.

Eligibility for Developed Vetting (DV) clearance is an essential requirement for this role.

Further information on eligibility and the clearance process can be found on the UK Government security vetting website.

Equality, Diversity and Inclusion

The Alan Turing Institute is committed to creating an environment where diversity is valued and everyone is treated fairly. We value diversity of background, experience, and perspective, and are proud to be an inclusive employer. We warmly encourage applications from all backgrounds, particularly from groups currently under‑represented in our sector. If you feel passionate about this role but don’t meet every single requirement, please apply - we recognise that great candidates may bring strengths beyond the criteria listed