Senior Data Engineer

Jobtailor

Naperville (IL)

On-site

USD 120,000 - 160,000

Full time

2 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Jobtailor is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines in a fast-paced healthcare data environment. You will develop ETL/ELT workflows using SQL, Python, and PySpark, handling clinical and claims data across multiple platforms.

The role emphasizes data quality, observability, and collaboration with Data Science, Product, and Infrastructure teams. You will optimize performance and contribute to a robust data platform architecture.

Qualifications

5+ years of professional Data Engineering experience.
Strong SQL skills with large datasets.
Hands-on Python and PySpark experience.
Experience with Apache Spark and distributed processing.
Experience with data platforms such as Databricks, Snowflake, BigQuery, Redshift.
Designing and operating production-grade ETL/ELT pipelines.
Data modeling, warehousing, and distributed data systems.
Linux command-line, bash/shell scripting.
Data quality, monitoring, and alerting frameworks.
Git and CI/CD experience.
Strong problem solving and cross-functional collaboration.
Nice to have: healthcare data standards (OMOP, FHIR, HL7, ICD, CPT).
Nice to have: AWS, GCP, or Azure experience.
Nice to have: Airflow, Dagster, or Prefect experience.
Nice to have: exposure to Generative AI and AI-enabled data applications.
Nice to have: experience in healthcare or data-intensive environments.

Responsibilities

Design, build, and maintain scalable, reliable data pipelines.
Develop ETL/ELT workflows with SQL, Python, PySpark, and Spark.
Work with healthcare datasets including clinical and claims data.
Build reliable ingestion and transformation workflows.
Improve data quality, monitoring, alerting, and observability.
Troubleshoot production pipeline issues using Linux and shell tools.
Optimize data processing performance and cloud costs.
Apply engineering practices for code quality, testing, and CI/CD.
Contribute to data modeling, warehousing, and distributed data architecture decisions.
Collaborate with Data Science, Clinical Informatics, Product, Infrastructure, and Commercial teams.
Support customer onboarding and complex data integration initiatives.
Participate in design and code reviews to improve engineering practices.
Identify opportunities to scale and improve the data platform.

Skills

SQL Proficiency

Python

PySpark

Apache Spark

Linux CLI

Git CI/CD

Data Modeling

Data Warehousing

Data Quality

Collaboration

Tools

Databricks

Snowflake

BigQuery

Redshift

Airflow

Dagster

Prefect

Job description

Design, build, and maintain scalable, reliable data pipelines
Develop high-performance ETL/ELT workflows using SQL, Python, PySpark, and Apache Spark
Work with complex healthcare datasets, including clinical and claims data
Build reliable, maintainable, and scalable ingestion and transformation workflows
Develop and improve data quality, monitoring, alerting, and observability solutions
Troubleshoot and resolve production data pipeline issues using Linux and shell-based tools
Optimize data processing performance and cloud infrastructure costs
Apply engineering practices around code quality, testing, version control, and CI/CD
Contribute to data modeling, data warehousing, and distributed data architecture decisions
Collaborate with Data Science, Clinical Informatics, Product, Infrastructure, and Commercial teams to translate requirements into data solutions
Support customer onboarding and complex data integration initiatives
Participate in design and code reviews and improve engineering practices
Identify opportunities to improve the scalability, reliability, and efficiency of the data platform

Requirements

5+ years of professional Data Engineering experience
Strong SQL skills and experience working with large datasets
Strong hands-on experience with Python and PySpark
Experience with Apache Spark and distributed data processing
Hands-on experience with modern data platforms such as Databricks, Snowflake, BigQuery, Redshift, or similar technologies
Experience designing and operating production-grade ETL/ELT pipelines
Strong understanding of data modeling, data warehousing, and distributed data systems
Solid Linux command-line experience, including bash and shell scripting
Experience implementing data quality, monitoring, and alerting frameworks
Experience with Git and CI/CD
Strong problem-solving skills and ability to independently investigate and resolve complex technical issues
Strong communication skills and ability to collaborate effectively with cross-functional teams
Nice to have: experience with healthcare data and standards such as OMOP, FHIR, HL7, ICD, CPT, claims, EHR/EMR, or related datasets
Nice to have: experience with AWS, GCP, or Azure
Nice to have: experience working with large-scale distributed systems
Nice to have: familiarity with Airflow, Dagster, Prefect, or similar workflow orchestration tools
Nice to have: exposure to Generative AI, LLMs, or AI-enabled data applications
Nice to have: experience working in healthcare, life sciences, health technology, or a data-intensive environment

Core Competencies

Demonstrates expertise in designing and maintaining scalable data pipelines, with strong proficiency in SQL, Python, and PySpark. Capable of optimizing data processing performance and implementing data quality frameworks in healthcare datasets.

Highest-signal resume keywords