Senior Data Engineer - Healthcare Data Pipelines

The Rawlings Group; United States; On-site; USD 180,000 - 220,000; Full time

Benefits offered by this job

Remote US

Healthcare

Unlimited PTO

Equity

401(k) including employer match

Flexible environment

Job summary

Machinify is seeking a Senior Data Engineer to transform raw external data into trusted datasets powering payment, product, and operations decisions. You will own end-to-end pipelines, onboard new customers, and ensure data is accurate, observable, and actionable.

You will design robust pipelines, canonicalize healthcare data, and drive data quality and observability across teams in a fast-moving environment. Remote US-friendly with competitive compensation.

Qualifications

• 6+ years of experience as a Data Engineer building production-grade pipelines.
• Strong expertise in Python, Spark SQL, and Airflow.
• Experience processing large-scale file-based datasets (CSV, Parquet, JSON) in production.
• Experience mapping and standardizing raw external data into canonical models.
• Familiarity with AWS (or any cloud) including file storage and distributed compute concepts.
• Experience onboarding new customers and integrating external customer data with non-standard formats.
• Ability to work across teams, manage priorities, and own complex data workflows with minimal supervision.
• Strong written and verbal communication skills.

Responsibilities

• Design and implement robust, production-grade pipelines using Python, Spark SQL, and Airflow to process high-volume file-based datasets (CSV, Parquet, JSON).
• Lead efforts to canonicalize raw healthcare data (837 claims, EHR, partner data, flat files) into internal models.
• Own the full lifecycle of core pipelines - from file ingestion to validated, queryable datasets.
• Onboard new customers by integrating their raw data into internal pipelines and canonical models; collaborate with SMEs, Account Managers, and Product to ensure successful implementation and troubleshooting.
• Build resilient, idempotent transformation logic with data quality checks, validation layers, and observability.
• Refactor and scale existing pipelines to meet growing data and business needs.
• Tune Spark jobs and optimized distributed processing performance.
• Implementschema enforcement and versioning aligned with internal data standards.
• Collaborate with Data Analysts, Data Scientists, Product Managers, Engineering, Platform, SMEs, and AMs to ensure pipelines meet evolving business needs.
• Monitor pipeline health, participate in on-call rotations, and proactively debug and resolve production data flow issues.
• Contribute to the evolution of our data platform - driving toward mature patterns in observability, testing, and automation.
• Build and enhance streaming pipelines (Kafka, SQS, or similar) where needed to support near-real-time data needs.

Skills

Python

Spark SQL

Airflow

Data pipelines

Cloud (AWS)

Tools

Kafka

SQS

Parquet

Job description

Machinify is seeking a Senior Data Engineer to transform raw external data into trusted datasets powering payment, product, and operations decisions. You will own end-to-end pipelines, onboard new customers, and ensure data is accurate, observable, and actionable.

You will design robust pipelines, canonicalize healthcare data, and drive data quality and observability across teams in a fast-moving environment. Remote US-friendly with competitive compensation.