Senior Data Engineer

Remote Worker LTD.; United States; Remote; USD 98,000 - 119,000; Full time

Benefits offered by this job

Competitive base salary

Generous pension scheme

30 days holiday + bank holidays

Parental leave

Cycle to work

Home & tech savings

EV scheme

Employee referral bonus

Wellbeing support

Central London office with flexible/ro

Job summary

Our Future Health, the UK's largest health research programme, is seeking a Senior Data Engineer with deep healthcare data expertise to design scalable pipelines and data products for research use. You will work across engineering, data science and epidemiology to deliver trusted datasets, and help set technical direction.

You will transform OMOP/FHIR data, implement quality controls, collaborate with researchers and product teams, and publish clear documentation while leveraging Azure, Spark,

Qualifications

• Proven data engineering experience in large-scale environments.
• Experience designing and maintaining scalable data pipelines from multiple sources.
• Experience with healthcare data across academia, healthcare, life sciences or health tech.
• Knowledge of OMOP and/or FHIR data models.
• Experience transforming data between CDMs.
• Understanding of healthcare terminologies (SNOMED CT, ICD-10).
• Proficiency in Python and software engineering best practices.
• Good command-line and Unix/Linux skills.
• Experience with Azure cloud, Spark, Databricks and data lake architectures.
• Experience with Docker/Kubernetes and Git workflows.
• Experience working in Agile delivery teams.

Responsibilities

• Designing and building scalable data pipelines ingesting healthcare data into trusted research environments.
• Developing data transformation logic in Python to create curated datasets.
• Contributing to shared codebases with testing, reviews and clean coding standards.
• Collaborating with data scientists, epidemiologists and product teams to deliver data solutions.
• Implementing data quality, validation and governance controls at scale.
• Transforming healthcare data to common data models like OMOP and FHIR.
• Creating reusable data products for multiple stakeholders.
• Producing clear technical documentation for pipelines and architectures.
• Supporting data flows across the healthcare data ecosystem.
• Leveraging Azure, Databricks, Spark and Data Lakes for high performance solutions.
• Promoting data standards and best practices across the team.
• Staying updated on trends in healthcare, research and data engineering.

Skills

Data engineering

Python

Unix/Linux

Agile delivery

Stakeholder collaboration

Education

Master's degree in Computer Science or related field

Tools

Azure

Spark

Databricks

Delta Lake

Parquet

Docker

Kubernetes

Git/GitHub

Job description

We're looking for a Senior Data Engineer with deep expertise in healthcare data to join a pioneering team building data products that enable world-leading research. Whether your experience comes from academia, healthcare, biotech, or a fast-growing health-tech startup, you'll bring a strong understanding of healthcare data standards such as OMOP or FHIR, alongside the technical leadership required to be a Senior Engineer.

In this role, you'll design and build scalable data pipelines that transform complex health records into trusted, research-ready datasets used by scientists and researchers tackling challenges with global significance. You'll work at the intersection of engineering, data science and epidemiology, helping to establish best practices, shape technical direction and create reusable, high-quality data products that can operate at industrial scale.

If you're a senior engineer who enjoys solving complex data challenges, influencing how healthcare data is curated and delivered, and making a meaningful impact on research that improves lives, we'd love to hear from you.

Our Future Health will be the UK’s largest ever health research programme, bringing people together to develop new ways to detect, prevent, and treat diseases. Diseases like cancer, dementia, diabetes, and heart disease affect the lives of many people in our communities. Our goal is to create a world-leading resource for health research, to improve our understanding and spot the patterns of how and why common diseases start, so treatments can begin sooner and be more effective.

Our plan is to bring together 5 million volunteers from right across the UK who will be asked to contribute information to help build one of the most detailed pictures we have ever had of people’s health. Researchers will be able to use this information to make new discoveries about human health and diseases. So future generations can live in good health for longer.

What you'll be doing:

• Designing and building scalable data pipelines that ingest healthcare data from multiple providers into trusted research environments and data platforms.
• Developing and maintaining data transformation logic in Python to create high-quality, curated and research-ready datasets.
• Contributing to complex, shared codebases while following engineering best practices including code reviews, testing and clean coding standards.
• Collaborating with data scientists, epidemiologists, software engineers and product teams to understand requirements and deliver impactful data solutions.
• Processing large-scale healthcare datasets and implementing robust data quality, validation and governance controls.
• Transforming and harmonising healthcare data into common data models such as OMOP and FHIR to support research and analytical use cases.
• Creating reusable, scalable and maintainable data products that enable efficient consumption by multiple stakeholders.
• Communicating effectively with technical and non-technical stakeholders to translate complex requirements into practical engineering solutions.
• Producing clear and comprehensive technical documentation covering data pipelines, transformation logic, architecture and operational processes.
• Supporting the design, scoping and implementation of data flows across the wider healthcare data ecosystem.
• Leveraging cloud and big data technologies, including Azure, Databricks, Spark and Data Lakes, to deliver high-performing data solutions.
• Adopting and promoting data engineering best practices, healthcare data standards and continuous improvement across the team.
• Keeping abreast of emerging trends, technologies and standards within healthcare, research and data engineering to drive innovation and excellence.

Requirements

To succeed in tis role you will need to demonstrate some of the following:

• Proven Data Engineering experience within complex, large-scale data environments.
• Experience designing, building and maintaining scalable, reliable data pipelines that process large volumes of data from multiple sources.
• Experience working with healthcare data across academia, healthcare, life sciences, health technology or related sectors.
• Knowledge of healthcare data standards and common data models, including OMOP and/or FHIR.
• Experience transforming and remodelling data between different Common Data Models (CDMs).
• Understanding of healthcare terminologies and ontologies, including SNOMED CT, ICD-10 and Read Codes.
• Experience contributing to shared codebases and developing reusable, maintainable data solutions.
• Proficiency in Python and software engineering best practices.
• Good command-line and Unix/Linux skills.
• Experience working with cloud platforms, ideally Microsoft Azure.
• Experience using technologies such as Spark, Databricks and data lake architectures.
• Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake.
• Experience with containerisation and orchestration technologies, including Docker and Kubernetes.
• Experience using Git/GitHub and working within collaborative development environments.
• Experience working within Agile delivery teams.

Benefits

• Competitive base salary from £74,000 per annum
• Generous Pension Scheme – We invest in your future with employer contributions of up to 12%
• 30 Days Holiday pro rata + Bank Holidays – Enjoy a generous holiday allowance with the flexibility to take bank holidays when it suits you
• Enhanced Parental Leave – Supporting you during life’s biggest moments
• Cycle to Work Scheme – Save 25-39% on a new bike and accessories through salary sacrifice
• Home & Tech Savings – Get up to 8% off on IKEA and Currys products, spreading the cost over 12 months through salary sacrifice
• EV Scheme – Save up to 40% on a brand new electric vehicle all-inclusive package through salary sacrifice
• £1,000 Employee Referral Bonus – Know someone amazing? Get rewarded for bringing them on board!
• Wellbeing Support – Access to Mental Health First Aiders, plus 24/7 online GP services and an Employee Assistance Programme for you and your family
• A Great Place to Work – We have a lovely Central London office in Holborn, and offer flexible and remote working arrangements

You may work remotely from anywhere within the UK, with occasional travel to London.

At Our Future Health, we recognise the importance of having a diverse workforce and ensuring that all candidates, regardless of their background, have equitable access to our application process. We proactively encourage applicants who identify as having a disability, neurodiversity, or long-term health conditions to let us know if they require any reasonable adjustments as part of their application process.

If you do require any reasonable adjustments, please email us at [e-mail]