Senior Data Engineer

machinifyinc; United States; Remote; USD 180,000 - 220,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Flexible time off

Equity

401(k) with employer match

Job summary

Machinify Inc. is seeking a Data Engineer to own production data workflows that turn raw external files into reliable datasets for product features, analytics, ML, and operational decisions.

You will handle customer data onboarding, data modeling, pipeline reliability, and ongoing improvements to a growing data platform. You will build and operate production pipelines using Python, Spark SQL, and Airflow to process large CSV, Parquet, and JSON datasets, map source data into canonical models,

Qualifications

• Six or more years in data engineering or a comparable role building production pipelines.
• Strong experience with Python, Spark SQL, and Airflow, plus large-scale file processing.
• Experience standardizing external data into canonical models and integrating customer data with varied formats.
• Familiarity with a cloud platform, including storage and distributed computing concepts.
• Ability to own complex workflows, balance priorities, and work across technical and non-technical teams.
• Clear written and verbal communication, including explaining technical decisions to non-engineering partners.

Responsibilities

• Build and operate production pipelines with Python, Spark SQL, and Airflow to process large CSV, Parquet, and JSON datasets.
• Map varied source data, including claims, EHR, partner feeds, and flat files, into consistent internal models.
• Manage pipeline workflows from ingestion through validation and queryable outputs, with attention to reliability and performance.
• Integrate new customer data, troubleshoot format and implementation issues, and coordinate with product and subject matter experts.
• Add idempotent transformations, quality checks, schema enforcement, versioning, and observability.
• Improve existing pipelines, tune distributed Spark processing, monitor production health, and take part in on-call support.
• Develop streaming workflows where needed and contribute to shared data engineering and modeling practices.

Skills

Data modeling

Cross-functional collaboration

Clear communication

Education

Six+ years data engineering experience

Tools

Python

Spark SQL

Airflow

Large-scale file processing

Job description

Role overview

Own production data workflows that turn raw external files into dependable datasets for product features, analytics, machine learning, and operational decisions. The role includes customer data onboarding, data modeling, pipeline reliability, and ongoing improvements to a growing data platform.

Responsibilities

• Build and operate production pipelines with Python, Spark SQL, and Airflow to process large CSV, Parquet, and JSON datasets.
• Map varied source data, including claims, EHR, partner feeds, and flat files, into consistent internal models.
• Manage pipeline workflows from ingestion through validation and queryable outputs, with attention to reliability and performance.
• Integrate new customer data, troubleshoot format and implementation issues, and coordinate with product and subject matter experts.
• Add idempotent transformations, quality checks, schema enforcement, versioning, and observability.
• Improve existing pipelines, tune distributed Spark processing, monitor production health, and take part in on-call support.
• Develop streaming workflows where needed and contribute to shared data engineering and modeling practices.

Requirements

• Six or more years in data engineering or a comparable role building production pipelines.
• Strong experience with Python, Spark SQL, and Airflow, plus large-scale file processing.
• Experience standardizing external data into canonical models and integrating customer data with varied formats.
• Familiarity with a cloud platform, including storage and distributed computing concepts.
• Ability to own complex workflows, balance priorities, and work across technical and non-technical teams.
• Clear written and verbal communication, including explaining technical decisions to non-engineering partners.

Nice to have

• Experience with healthcare data such as 837 or 835 claims, EHR, UB04, or claims normalization.
• Familiarity with Kafka or SQS is useful; willingness to learn streaming systems is also considered.

Benefits and work setup

The position can be performed from anywhere in the US. Listed offerings include medical, dental, and vision coverage, flexible time off, equity, and a 401(k) with employer match. The stated base salary range is $180,000-$220,000.