Data Engineer - Healthcare

Sand Technologies

United States

On-site

USD 120,000 - 180,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Sand Technologies in the United States is seeking a Data Engineer to own ingestion and transformation of HL7 v2, C-CDA, FHIR R4 and claims data into a unified health data ontology on our Symmetri platform running in AWS. You will ensure raw, replayable data and strong provenance while supporting multi-wave migrations in a regulated healthcare context.

The role emphasizes testable, production-grade pipelines, observable systems, and collaboration with clients to meet tight federal deadlines.

Qualifications

  • 5+ years designing and operating large-scale data platforms in cloud or hybrid environments.
  • Strong proficiency in Python and SQL with production-quality code.
  • Experience with distributed processing (Spark/PySpark) and streaming systems (Kafka).
  • Experience building lakehouse architectures on open table formats with transactions and time travel.
  • Experience deploying and operating on AWS; CI/CD and infrastructure-as-code; Kubernetes helpful.
  • Ability to design highly available, observable data systems with SLAs and SLOs.

Responsibilities

  • Build the ingestion layer: pipelines for HL7 v2, C-CDA, FHIR R4 and claims files.
  • Land data in raw, replayable form with provenance: source, time, version.
  • Map sources to one ontology: Person, Encounter, Condition, Result, Medication, Claim.
  • Run migration waves: shadow load, parallel run, cutover with rollback.
  • Prove data moved correctly with row-level and record-level reconciliation.
  • Make data quality measurable per sender and generate six-dimension scorecards.
  • Engineer for failure: retries, dead-letter queue, circuit breaker and audit trails.
  • Profile pipelines for latency and data quality; rebuild with validation at intake.
  • Write data contracts: schema, quality targets, ownership, refresh and consumers.

Skills

Python
SQL
dbt
Spark/PySpark
Kafka
AWS
Kubernetes
CI/CD
Data pipelines
Distributed processing

Tools

dbt
Mirth Connect
Verato

Job description

Data Engineer
About Sand

Sand Technologies is a global Physical AI company using data and AI to make critical industries work better. We partner with governments, cities and enterprises to improve how essential systems operate across healthcare, water, energy, telecommunications and infrastructure.

Our work delivers proven real-world impact. We have built AI systems that help manage London's water supply, supported telecom network planning across hundreds of cities, and developed digital healthcare platforms serving tens of millions of people across Africa. From intelligent command centers to AI-powered infrastructure platforms, we help organizations sense, analyze and act in complex environments.

Our people are ambitious, curious and relentlessly practical. Our teams work alongside clients in the field, solving hard problems and deploying solutions that last. With colleagues across Africa, Europe, the UK and the US, we operate across the full stack - from research and engineering to deployment and capability building.

Our mission is simple: to harness AI to solve humanity's most pressing challenges.

About the role

Delaware’s designated health information exchange and Health Data Utility is bringing its clinical data repository, its All Payer Claims Database and its internal warehouse onto one governed, longitudinal record. Sand is delivering that platform on Symmetri, our data platform, running inside the client’s own AWS account.

The scale is real. The exchange has moved clinical data since 2007 and claims since 2013. It handles roughly 4.7 million HL7 messages and 220,000 CCDs a month from about 120 data senders, holds claims covering about 87% of Delawareans, and maintains around 5.5 million patient identities across 611 sources.

None of it can stop while we move it. Results delivery, admission alerts and state claims reporting run throughout. We migrate source by source, in parallel, with reconciliation the client can see and a rehearsed rollback on every cutover.

You will own the pipelines and the migration waves that make this happen: building the ingestion and transformation layer for HL7 v2, C-CDA FHIR R4 and claims flat files, mapping them onto one health data ontology and proving - record by record - that what landed in the new platform matches what left the old one.

We operate with a strong code-first, \"data as a product\" mindset, where testing, reliability, observability and performance are non-negotiables. In a regulated health environment with fixed federal funding deadlines, that discipline is the job, not a nice-to-have.

What you’ll do
  • Build the ingestion layer. Design and operate pipelines for HL7 v2 (ADT, ORU, ORM, MDM, VXU), C-CDA, FHIR R4 and claims flat files, parsing documents and messages into discrete, coded data elements rather than stored blobs.
  • Land everything raw and replayable. Every message and file is stored unchanged, time-stamped and re-runnable, with element-level provenance: source system, sender, received time, original message, pipeline and mapping version.
  • Map sources to one ontology. Transform records into shared objects - Person, Encounter, Condition, Result, Medication, Claim - aligned to FHIR R4 and US Core on the clinical side and the APCD Common Data Layout on the claims side. Map local codes to ICD-10, CPT, LOINC, RxNorm and SNOMED CT.
  • Run the migration waves. Shadow load, parallel run, cutover with rollback, retire after acceptance. Four waves across claims history, clinical history, live feeds and outbound consumers, each one signed off by the client before anything is switched off.
  • Prove the data moved correctly. Build the row-level and record-level reconciliation that compares counts, totals, identities and sampled clinical content between old and new. These reports are reviewed by the client's data quality staff and gate every cutover.
  • Make data quality measurable per sender. Validate structure, required fields, code validity, duplicates and volumes against each sender's baseline, and produce six-dimension quality scorecards that go back to the source system owners.
  • Engineer for failure. Every pipeline ships with retries, a dead-letter queue and a circuit breaker. A record that fails a quality check is never silently dropped - it goes to a steward, who fixes and replays it or discards it with an audit record.
  • Optimize what exists. Profile current pipelines for latency, failure rate, rejected messages and data quality, establish baselines, and rebuild them with validation at intake.
  • Write the data contracts. Every migrated dataset carries its schema, quality targets, owner, refresh schedule and consumers.
Who you are

Core engineering

  • 5+ years designing and operating large-scale data platforms in cloud or hybrid environments, with production ownership rather than project handover.
  • Strong proficiency in Python and SQL, ideally with dbt, and a track record of clean, tested, production-quality code.
  • Deep hands-on expertise with distributed processing (Spark/PySpark) and streaming or event-driven systems (Kafka or equivalent).
  • Experience building lakehouse architectures on open table formats, with transactions, schema evolution and time travel.
  • Experience deploying and operating on AWS, including CI/CD and infrastructure-as-code. Kubernetes experience is valuable.
  • Demonstrated ability to design highly available, observable, production-grade data systems with defined SLAs and SLOs.

Migration and data quality

  • Proven experience migrating production data between platforms without interrupting live services - shadow loads, parallel runs, reconciliation and rehearsed rollback.
  • Fluency in reconciliation technique: row counts, checksums, record-level comparison, and the judgement to know when a mismatch is a real defect rather than a modelling difference.
  • Experience building automated data quality frameworks, lineage capture and alerting as part of the pipeline, not bolted on afterwards.

Health data and regulated environments

  • Working knowledge of health interoperability standards - HL7 v2, C-CDA, FHIR R4 and US Core. Experience with IHE profiles (XDS.b, XCA, PIX, PDQ) is a strong advantage.
  • Familiarity with clinical and claims terminology: ICD-10, CPT, LOINC, RxNorm, SNOMED CT, and value set management.
  • Experience with healthcare claims data, ideally All Payer Claims Database work or the APCD Common Data Layout.
  • Experience building secure, governed data environments in regulated settings, with working knowledge of HIPAA and HITECH. Exposure to 42 CFR Part 2, Data Segmentation for Privacy (DS4P) or consent enforcement is highly valued.
  • Experience working with an enterprise master patient index and identity resolution. Verato, Mirth Connect or comparable integration engines are a plus.

How you operate

  • Comfortable in multi-stakeholder environments where a public agency, incumbent vendors, payers and more than a hundred data senders all have a say.
  • Systems-thinking mindset, focused on scalability, modularity and long-term platform evolution.
  • Able to work to hard external deadlines and acceptance gates without trading away test coverage or auditability.

We would rather hear from you than not. If you are strong on data engineering and migration but have not yet worked with health standards, say so and apply. Health interoperability is learnable by a good engineer; the migration discipline and the regulated-environment instincts are much harder to teach.

Location

This role is not a remote position. You must be based in the United States and able to travel to the client site in Dover, Delaware as the project requires, including for design workshops and cutover windows. Travel is heaviest during Discovery and around production cutovers rather than on a fixed weekly pattern.

How we work

Due to the highly collaborative and internationally distributed nature of our work, successful candidates must be comfortable operating in small teams while contributing to larger, globally coordinated efforts. A strong sense of ownership, self-motivation and discipline in maintaining clear and consistent communication through virtual collaboration tools and video conferencing is essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer - Healthcare
Data Engineer - Healthcare

Sand Tech Holdings Limited • United States

On-site
USD 120,000 - 170,000
Data Engineer
Data Engineer

Sand Tech Holdings Limited • Baltimore (MD)

On-site
USD 110,000 - 180,000
Site Reliability Engineer
Site Reliability Engineer

SAND • United States

On-site
USD 140,000 - 210,000
Technical Lead
Technical Lead

SAND • United States

Hybrid
USD 120,000 - 190,000
Production AI Technical Lead — Cross-Stack Delivery
Production AI Technical Lead — Cross-Stack Delivery

SAND • United States

Hybrid
USD 120,000 - 190,000
Site Reliability Engineer
Site Reliability Engineer

sandtechholdingslimited • United States

On-site
USD 120,000 - 180,000
Senior Forward Deployed Data Engineer, Data Modernizaton
Senior Forward Deployed Data Engineer, Data Modernizaton

Qualified Health • United States

Hybrid
USD 180,000 - 230,000
Equity packages
Medical, dental, vision insurance
Unlimited PTO
+1
Data Engineer
Data Engineer

Claritas Rx • United States

Remote
USD 120,000 - 140,000
Unlimited PTO
Stock options
Flexible work environment
Healthcare Data Engineer: HL7/FHIR Pipelines & Migration
Healthcare Data Engineer: HL7/FHIR Pipelines & Migration

Sand Tech Holdings Limited • United States

On-site
USD 120,000 - 170,000
Research Consultant Data Scientist
Research Consultant Data Scientist

Dandelion Health AI • New York (NY)

Remote
USD 120,000 - 200,000
Remote work and flexible hours
Health, dental, vision benefits
PTO and sick days
+1