Senior Data Engineer

Our Future Health; Greater London; On-site; GBP 75,000 - 110,000; Full time

Job summary

PVH is seeking a Senior Data Engineer to design and build scalable data pipelines ingesting healthcare data from multiple providers into trusted research environments, transforming records into curated datasets for scientists and researchers.

You'll work across engineering, data science and epidemiology, applying OMOP/FHIR standards, Azure Databricks, Spark, and data lake architectures, while promoting best practices and maintainable data products.

Qualifications

• Experience designing scalable data pipelines for large healthcare datasets.
• Hands-on experience with OMOP and/or FHIR data models.
• Proficiency in Python and software engineering best practices.
• Ability to collaborate with researchers, epidemiologists and engineers.

Responsibilities

• Design and build scalable data pipelines ingesting healthcare data from multiple providers into trusted research environments.
• Transform and harmonise data to common data models (OMOP/FHIR) for research use cases.
• Contribute to shared codebases with tests, reviews and documentation.
• Collaborate with data scientists, epidemiologists and product teams to deliver data solutions.
• Ensure data quality, governance and robust validation across datasets.
• Leverage Azure, Databricks, Spark and Data Lakes to deliver performant solutions.

Skills

Python

Azure

Databricks

Spark

ETL pipelines

Data governance

OMOP/FHIR

Big data

Git/GitHub

Unix/Linux

Docker

Kubernetes

Healthcare data

Tools

Azure Data Lake

Parquet/Delta Lake

Docker

Kubernetes

Job description

Overview

We\'re looking for a Senior Data Engineer with deep expertise in healthcare data to join a pioneering team building data products that enable world-leading research. Whether your experience comes from academia, healthcare, biotech, or a fast-growing health-tech startup, you\'ll bring a strong understanding of healthcare data standards such as OMOP or FHIR, alongside the technical leadership required to be a Senior Engineer. In this role, you\'ll design and build scalable data pipelines that transform complex health records into trusted, research-ready datasets used by scientists and researchers tackling challenges with global significance. You\'ll work at the intersection of engineering, data science and epidemiology, helping to establish best practices, shape technical direction and create reusable, high-quality data products that can operate at industrial scale. If you\'re a senior engineer who enjoys solving complex data challenges, influencing how healthcare data is curated and delivered, and making a meaningful impact on research that improves lives, we\'d love to hear from you.

Our Future Health will be the UK\'s largest ever health research programme, bringing people together to develop new ways to detect, prevent, and treat diseases. Diseases like cancer, dementia, diabetes, and heart disease affect the lives of many people in our communities. Our goal is to create a world-leading resource for health research, to improve our understanding and spot the patterns of how and why common diseases start, so treatments can begin sooner and be more effective. Our plan is to bring together 5 million volunteers from right across the UK who will be asked to contribute information to help build one of the most detailed pictures we have ever had of people\'s health. Researchers will be able to use this information to make new discoveries about human health and diseases. So future generations can live in good health for longer.

What you\'ll be doing

• Designing and building scalable data pipelines that ingest healthcare data from multiple providers into trusted research environments and data platforms.
• Developing and maintaining data transformation logic in Python to create high-quality, curated and research-ready datasets.
• Contributing to complex, shared codebases while following engineering best practices including code reviews, testing and clean coding standards.
• Collaborating with data scientists, epidemiologists, software engineers and product teams to understand requirements and deliver impactful data solutions.
• Processing large-scale healthcare datasets and implementing robust data quality, validation and governance controls.
• Transforming and harmonising healthcare data into common data models such as OMOP and FHIR to support research and analytical use cases.
• Creating reusable, scalable and maintainable data products that enable efficient consumption by multiple stakeholders.
• Communicating effectively with technical and non-technical stakeholders to translate complex requirements into practical engineering solutions.
• Producing clear and comprehensive technical documentation covering data pipelines, transformation logic, architecture and operational processes.
• Supporting the design, scoping and implementation of data flows across the wider healthcare data ecosystem.
• Leveraging cloud and big data technologies, including Azure, Databricks, Spark and Data Lakes, to deliver high-performing data solutions.
• Adopting and promoting data engineering best practices, healthcare data standards and continuous improvement across the team.
• Keeping abreast of emerging trends, technologies and standards within healthcare, research and data engineering to drive innovation and excellence.

Qualifications / What you\'ll need to succeed

• Proven Data Engineering experience within complex, large-scale data environments.
• Experience designing, building and maintaining scalable, reliable data pipelines that process large volumes of data from multiple sources.
• Experience working with healthcare data across academia, healthcare, life sciences, health technology or related sectors.
• Knowledge of healthcare data standards and common data models, including OMOP and/or FHIR.
• Experience transforming and remodelling data between different Common Data Models (CDMs).
• Understanding of healthcare terminologies and ontologies, including SNOMED CT, ICD-10 and Read Codes.
• Experience contributing to shared codebases and developing reusable, maintainable data solutions.
• Proficiency in Python and software engineering best practices.
• Good command-line and Unix/Linux skills.
• Experience working with cloud platforms, ideally Microsoft Azure.
• Experience using technologies such as Spark, Databricks and data lake architectures.
• Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake.
• Experience with containerisation and orchestration technologies, including Docker and Kubernetes.
• Experience using Git/GitHub