Senior Software Engineer (Core Data Services)

TN United Kingdom; London; On-site; GBP 85,000 - 95,000; Full time

Job summary

Join a forward-thinking company as a Senior Software Engineer in London, where you'll work on innovative data processing solutions for health research. This dynamic role involves developing and enhancing features, collaborating with cross-functional teams, and contributing to high-quality code. You'll have the opportunity to work with cutting-edge technologies, including cloud-native data processing and orchestration systems. With a commitment to improving health outcomes, this role offers a chance to make a meaningful impact while enjoying flexible working arrangements and a competitive benefits package.

Benefits

Pension contributions up to 12%

30 days holiday plus bank holidays

Parental leave

Learning budget

Cycle to work scheme

Discounts

Referral bonus

Wellbeing support

Flexible working

Qualifications

• Experience with open source data orchestration systems like Dagster or Prefect.
• Proficient in distributed compute engines such as Spark or Databricks.

Responsibilities

• Developing new features and improving existing ones.
• Collaborating through code reviews and discussions.
• Supporting live operations and managing incidents.

Skills

Python

PySpark

Databricks

Kubernetes

Postgres

Dagster

Azure

CI/CD

Git

Agile practices

Tools

Docker

Terraform

Helm

Job description

Social network you want to login/join with:

Senior Software Engineer (Core Data Services), London

Client: Our Future Health

Location: London, United Kingdom

Job Category: -

EU work permit required: Yes

Job Reference: ead06191a1a0

Job Views: 3

Posted: 26.04.2025

Expiry Date: 10.06.2025

Job Description:

We are hiring a Senior Software Engineer to join our Core Data Services squad. This team owns our shared data release assets and builds tooling that enables work across research and data platforms.

The tooling supports processing and storage of high-volume health, NHS, and genetic datasets, and their release to Trusted Research Environments (TREs).

We seek candidates experienced in Python and cloud-native data processing, especially PySpark/Databricks, K8s, Postgres, Dagster, and Azure. We welcome those eager to expand their knowledge. Candidates should be passionate, produce high-quality code, and be adept at troubleshooting.

Our Future Health aims to be the UK’s largest health research program, involving 5 million volunteers to advance understanding of health and disease, supported by the UK Government, charities, and industry, working closely with the NHS.

What You’ll Be Doing

• Developing new features, improving existing ones, managing tech debt, and enhancing code quality.
• Collaborating through code reviews, feedback, and discussions to uphold high standards.
• Refining requirements and proposing architecture for upcoming initiatives.
• Participating in retrospectives and suggesting improvements.
• Working with internal and platform squads using shared tooling.
• Supporting live operations, monitoring systems, and managing incidents.
• Contributing to engineering initiatives to improve workflows for Our Future Health.

Requirements

Ideal candidates will have experience with:

• Open source data orchestration systems like Dagster, Prefect, or Airflow
• Distributed compute engines such as Spark/Databricks
• Docker, Kubernetes, Helm in cloud environments
• Building software in Azure (or AWS/GCP)
• Python package development
• Data modeling, relational databases (PostgreSQL), and other data stores
• HTTP APIs, scaling, and authentication protocols
• Agile practices, unit testing, CI/CD, code reviews, Git, GitHub
• Operational management including observability, logging, metrics, incident management
• Handling sensitive data, GDPR, ISO 27001, regulated environments
• Terraform and Infrastructure as Code (IaC)

Benefits include a salary of £85,000–£95,000, pension contributions up to 12%, 30 days holiday plus bank holidays, parental leave, learning budget, cycle to work, discounts, referral bonus, wellbeing support, and flexible working in London’s Holborn.