Staff Data Engineer — Spark & Healthcare Data (Remote)

H1; York and North Yorkshire; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Flexible hours

Stock options

Health & life insurance

Unlimited PTO

Work from home

Job summary

H1 is seeking a Staff Data Engineer for the EMERALD team to shape architecture and scale for its healthcare entity resolution platform. You will lead a small team, stay hands-on, and own services powering automatching, identity mapping, grouping, deduplication, and enrichment across tens of millions of records.

You will collaborate with Product, AI/ML, Analytics, and Engineering to improve accuracy, reliability, and efficiency in a cloud-native environment using Spark, AWS, and modern infra

Qualifications

• Strong hands-on data engineering background with distributed systems.
• Experience building large-scale Spark/PySpark pipelines.

Responsibilities

• Lead design, scalability, and technical direction of the EMERALD platform.
• Own automatching, identity mapping, deduplication, and enrichment workflows.
• Mentor engineers, review code, and set engineering best practices.
• Collaborate with Product/AI/ML to improve matching precision and recall.

Skills

PySpark

Scala/Java

Apache Spark

AWS

Kubernetes

Terraform

Docker

ETL/ELT

Entity resolution

Distributed systems

Tools

Airflow

Job description

H1 is seeking a Staff Data Engineer for the EMERALD team to shape architecture and scale for its healthcare entity resolution platform. You will lead a small team, stay hands-on, and own services powering automatching, identity mapping, grouping, deduplication, and enrichment across tens of millions of records.

You will collaborate with Product, AI/ML, Analytics, and Engineering to improve accuracy, reliability, and efficiency in a cloud-native environment using Spark, AWS, and modern infra