Senior Data Engineer: Scalable Pipelines, Healthcare Data

Jobtailor; Naperville (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

Jobtailor is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines in a fast-paced healthcare data environment. You will develop ETL/ELT workflows using SQL, Python, and PySpark, handling clinical and claims data across multiple platforms.

The role emphasizes data quality, observability, and collaboration with Data Science, Product, and Infrastructure teams. You will optimize performance and contribute to a robust data platform architecture.

Qualifications

• 5+ years of professional Data Engineering experience.
• Strong SQL skills with large datasets.
• Hands-on Python and PySpark experience.
• Experience with Apache Spark and distributed processing.
• Experience with data platforms such as Databricks, Snowflake, BigQuery, Redshift.
• Designing and operating production-grade ETL/ELT pipelines.
• Data modeling, warehousing, and distributed data systems.
• Linux command-line, bash/shell scripting.
• Data quality, monitoring, and alerting frameworks.
• Git and CI/CD experience.
• Strong problem solving and cross-functional collaboration.
• Nice to have: healthcare data standards (OMOP, FHIR, HL7, ICD, CPT).
• Nice to have: AWS, GCP, or Azure experience.
• Nice to have: Airflow, Dagster, or Prefect experience.
• Nice to have: exposure to Generative AI and AI-enabled data applications.
• Nice to have: experience in healthcare or data-intensive environments.

Responsibilities

• Design, build, and maintain scalable, reliable data pipelines.
• Develop ETL/ELT workflows with SQL, Python, PySpark, and Spark.
• Work with healthcare datasets including clinical and claims data.
• Build reliable ingestion and transformation workflows.
• Improve data quality, monitoring, alerting, and observability.
• Troubleshoot production pipeline issues using Linux and shell tools.
• Optimize data processing performance and cloud costs.
• Apply engineering practices for code quality, testing, and CI/CD.
• Contribute to data modeling, warehousing, and distributed data architecture decisions.
• Collaborate with Data Science, Clinical Informatics, Product, Infrastructure, and Commercial teams.
• Support customer onboarding and complex data integration initiatives.
• Participate in design and code reviews to improve engineering practices.
• Identify opportunities to scale and improve the data platform.

Skills

SQL Proficiency

Python

PySpark

Apache Spark

Linux CLI

Git CI/CD

Data Modeling

Data Warehousing

Data Quality

Collaboration

Tools

Databricks

Snowflake

BigQuery

Redshift

Airflow

Dagster

Prefect

Job description

Jobtailor is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines in a fast-paced healthcare data environment. You will develop ETL/ELT workflows using SQL, Python, and PySpark, handling clinical and claims data across multiple platforms.

The role emphasizes data quality, observability, and collaboration with Data Science, Product, and Infrastructure teams. You will optimize performance and contribute to a robust data platform architecture.