Senior Data Engineer - Big Data Pipelines & APIs

H1; New York (NY); On-site; USD 185,000 - 230,000; Full time

Benefits offered by this job

Health insurance options

Paid time off

Wellness holidays

Retirement options

RSU equity

Employee resource groups

Job summary

H1 is seeking a senior data engineering professional to design and operate large-scale data pipelines, focusing on ingestion, transformation, matching and synchronization of client data. You will optimize the entity matching service, improve data quality, and build APIs powering customer-facing UIs and internal tools.

You will work directly with enterprise data, collaborate with product and design teams, and contribute to architecture decisions while advancing the platform for healthcare data

Qualifications

• 8+ years of software engineering experience with a focus on data engineering and building data-driven products.
• Experience designing, building and running production big data pipelines at scale (ETL/ELT, orchestration, monitoring).
• Strong proficiency in PySpark, Scala, Python and SQL.
• Solid experience with distributed data processing and performance tuning in Spark.
• Experience building APIs or backend services supporting customer-facing applications.
• Working knowledge of cloud infrastructure (AWS, GCP or Azure).
• Hands-on experience with entity resolution, record matching or master data management.
• Experience with data quality, scoring or validation frameworks.
• Comfort working directly with client data, including messy source files, and onboarding enterprise customers.
• Product-engineering mindset: care about how customers use what you build.
• Preferred: full-stack experience or React for web UI.

Responsibilities

• Design, build and operate large-scale data pipelines for ingesting, transforming, matching and syncing client data using PySpark, Scala and SQL.
• Build and improve the entity matching service against H1's core data layer.
• Develop data asset management features such as scoring, status assignment and configurable intake.
• Build APIs and backend services powering customer-facing UIs and internal tools.
• Contribute to customer-facing and internal applications to bring new products to market.
• Build self-service internal tools for operations and customer success teams.
• Work with messy enterprise client data, debugging issues end-to-end and improving ingestion reliability.
• Integrate clinical and provider data as new products and customers come onto the platform.
• Collaborate with product and design partners to shape features from problem to shipped product.
• Improve pipeline performance, reliability, observability, testing and documentation.
• Contribute to architecture decisions and code reviews.
• Collaborate with adjacent teams owning feed delivery and roster processing.

Skills

Big data engineering

Data pipelines

Spark performance tuning

PySpark

Scala

Python

SQL

APIs/backend services

Cloud platforms

Entity resolution

Data quality

Product mindset

Full-stack / React

Tools

AWS

GCP

Azure

React

Job description

H1 is seeking a senior data engineering professional to design and operate large-scale data pipelines, focusing on ingestion, transformation, matching and synchronization of client data. You will optimize the entity matching service, improve data quality, and build APIs powering customer-facing UIs and internal tools.

You will work directly with enterprise data, collaborate with product and design teams, and contribute to architecture decisions while advancing the platform for healthcare data