Senior Data Engineer

Government Recruitment Service; Greater London; On-site; GBP 64,000 - 85,000; Full time

Job summary

Government Recruitment Service in London is seeking a Senior Data Engineer to design, build and run scalable data products. You will lead data pipelines using SQL, Python and AWS, and work with cross-functional teams.

You will mentor junior engineers and promote best practices while delivering robust data solutions for government services.

Qualifications

• Experience using SQL, Python and AWS to build robust data products.
• Ability to coach and develop junior team members.
• Strong collaboration with stakeholders in cross-functional, agile teams.

Responsibilities

• Lead the design, build, and testing of data products by integrating feeds from multiple systems.
• Ensure resilient, scalable data solutions using appropriate technologies.
• Troubleshoot database and data process issues with standard approaches.
• Collaborate with stakeholders to understand requirements and deliverables.
• Provide user support and problem resolution for data tools.

Skills

SQL

Python

AWS

Data pipelines

Tools

Apache Airflow

Terraform

PostgreSQL

Job description

The Department for Business, Innovation, Science and Trade (BIST) helps businesses invest, innovate, export and grow across the UK. The Digital, Data and Technology (DDaT) directorate helps to deliver this by designing, building and running the services, platforms and technology that businesses and colleagues rely on every day.

Our teams work on a wide range of services and technologies, including:

• digital services that help businesses access government support, guidance and opportunities
• data and AI products that support decision making and enable innovation
• internal tools and services that help colleagues work more effectively
• secure technology and platforms that support critical government services
• cyber security services that protect systems, data and users

By joining DDaT, you'll work on services used by businesses, colleagues and citizens across the UK. You'll be part of multidisciplinary teams focused on improving services, solving complex problems and delivering better outcomes for users.

We are committed to creating an inclusive and supportive workplace. In recognition of this, we were named Best Public Sector Employer at the Women in Tech Employer Awards 2025!

Data engineers at BIST are pivotal to our mission of driving economic growth by placing data at the heart of decision-making. We leverage leading cloud and open-source technology to build scalable, business-critical data pipelines, products and services, ensuring our users—whether within BIST, across government, or the public— have access to the right data in the optimal format.

This could include:

Using Apache Airflow and various AWS tools to build robust, scalable pipelines

Using Terraform to deploy and maintain our Postgres-backed data platform

Our cross-functional, agile teams work collaboratively on a range of projects from implementing AI tools, to our data science and analytics platform, to our system for managing the British Industrial Competitiveness Scheme (BICs), and beyond. Data Engineers also belong to a vibrant community (both within DBT and across government) who share and promote best practice.

We are seeking a collaborative and innovative problem-solver to help us deliver better outcomes for our users. You will use your expertise in SQL, Python and AWS to build robust data products, develop scalable data products. You will also have experience in supporting and developing more junior members of the team.

Main responsibilities

You will:

• Lead the design, build, and testing of data products: Develop robust data products by integrating feeds from multiple systems, utilising a variety of storage technologies and access methods
• Ensure the delivery of resilient and scalable solutions: Select and deploy the most suitable technologies to create data solutions that are resilient, scalable, and future proof
• Have problem-solving expertise: Understanding of common issues in databases, data processes, products, and services, and apply standard solutions effectively
• Collaborate with stakeholders: Work closely with a diverse range of stakeholders to understand their requirements, establish project goals, and define deliverables.
• Support users and solve issues: Assist users in utilising data tools, resolve issues promptly, and elevate when necessary
• Promote best practices: Develop and advocate for best practices within the team.
• Encourage team building and development: Organise initiatives to foster team building and professional development
• Provide line management and support to junior team members, guiding their growth and development
• Drive continuous improvement: Stay informed about the latest trends and best practices in data engineering to drive continuous improvement

Proud member of the Disability Confident employer scheme

Senior Data Engineer

Company: Government Recruitment Service

Salary: £64,332 to £84,826 a year

Hours: Full-time

Location: London

Job type: Permanent

Posting date: 9 Nov 2026

Closing date: 27 Nov 2026

Summary

The Department for Business, Innovation, Science and Trade (BIST) helps businesses invest, innovate, export and grow across the UK. The Digital, Data and Technology (DDaT) directorate helps to deliver this by designing, building and running the services, platforms and technology that businesses and colleagues rely on every day.

Our teams work on a wide range of services and technologies, including:

• digital services that help businesses access government support, guidance and opportunities
• data and AI products that support decision making and enable innovation
• internal tools and services that help colleagues work more effectively
• secure technology and platforms that support critical government services
• cyber security services that protect systems, data and users

By joining DDaT, you'll work on services used by businesses, colleagues and citizens across the UK. You'll be part of multidisciplinary teams focused on improving services, solving complex problems and delivering better outcomes for users.

We are committed to creating an inclusive and supportive workplace. In recognition of this, we were named Best Public Sector Employer at the Women in Tech Employer Awards 2025!

Data engineers at BIST are pivotal to our mission of driving economic growth by placing data at the heart of decision-making. We leverage leading cloud and open-source technology to build scalable, business-critical data pipelines, products and services, ensuring our users—whether within BIST, across government, or the public— have access to the right data in the optimal format.

This could include:

Using Apache Airflow and various AWS tools to build robust, scalable pipelines

Using Terraform to deploy and maintain our Postgres-backed data platform

Our cross-functional, agile teams work collaboratively on a range of projects from implementing AI tools, to our data science and analytics platform, to our system for managing the British Industrial Competitiveness Scheme (BICs), and beyond. Data Engineers also belong to a vibrant community (both within DBT and across government) who share and promote best practice.

We are seeking a collaborative and innovative problem-solver to help us deliver better outcomes for our users. You will use your expertise in SQL, Python and AWS to build robust data products, develop scalable data products. You will also have experience in supporting and developing more junior members of the team.

Main responsibilities

You will:

• Lead the design, build, and testing of data products: Develop robust data products by integrating feeds from multiple systems, utilising a variety of storage technologies and access methods
• Ensure the delivery of resilient and scalable solutions: Select and deploy the most suitable technologies to create data solutions that are resilient, scalable, and future proof
• Have problem-solving expertise: Understanding of common issues in databases, data processes, products, and services, and apply standard solutions effectively
• Collaborate with stakeholders: Work closely with a diverse range of stakeholders to understand their requirements, establish project goals, and define deliverables.
• Support users and solve issues: Assist users in utilising data tools, resolve issues promptly, and elevate when necessary
• Promote best practices: Develop and advocate for best practices within the team.
• Encourage team building and development: Organise initiatives to foster team building and professional development
• Provide line management and support to junior team members, guiding their growth and development
• Drive continuous improvement: Stay informed about the latest trends and best practices in data engineering to drive continuous improvement

Proud member of the Disability Confident employer scheme

About Disability Confident

Disability Confident is a government scheme. It encourages employers to recruit and retain disabled people and those with long term health conditions.

Related jobs

Lead Site Reliability Engineer

£68,732 to £88,423 per year

Government Recruitment Service

London

Permanent Full time

Digital Performance Analyst

£44,913 to £53,230 per year

Government Recruitment Service

London

Permanent Full time

Data Scientist

Hays Specialist Recruitment

London, London

Contract Full time

Browse more jobs