Senior Azure Data Engineer: Real-Time Pipelines & Databricks

Motion Recruitment; Chicago (IL); Hybrid; USD 140,000 - 190,000; Full time

Job summary

A leading healthcare performance improvement firm is seeking a Sr. Azure Data Engineer in Chicago. The ideal candidate will have over 5 years of experience in data engineering and strong skills in designing and supporting data pipelines. Proficiency in Azure Data Factory, PySpark, and SQL is essential. This role requires hands-on experience with cloud environments and a commitment to delivering quality results. This position supports a hybrid work model with three days onsite.

Qualifications

• 5+ years' experience as a data engineer.
• 3+ years with ETL/ELT concepts and troubleshooting.
• 2+ years of production-quality Python development.
• 1+ years using Azure Data Factory (ADF) and Databricks.

Responsibilities

• Design and support end-to-end data pipelines.
• Optimize SQL and PySpark transformations.
• Create and maintain Azure Data Factory pipelines.
• Provision Azure components using Pulumi.

Skills

Data engineering

ETL/ELT concepts

SQL advanced querying

Python development

PySpark optimization

Azure Data Factory

Databricks workflows

Infrastructure as Code (Pulumi)

Tools

Azure SQL

ADLS Gen2

Python testing frameworks

Pulumi

Job description

A leading healthcare performance improvement firm is seeking a Sr. Azure Data Engineer in Chicago. The ideal candidate will have over 5 years of experience in data engineering and strong skills in designing and supporting data pipelines. Proficiency in Azure Data Factory, PySpark, and SQL is essential. This role requires hands-on experience with cloud environments and a commitment to delivering quality results. This position supports a hybrid work model with three days onsite.