Senior Data Engineer

Harnham; Chicago (IL); Hybrid; USD 150,000 - 190,000; Full time

Job summary

Harnham in Chicago is seeking an experienced Data Engineer to help lead a major Data & AI modernization initiative. You will shape a cloud-based data platform, build production-grade data pipelines, and establish engineering standards in a growing team.

The role emphasizes Azure Databricks, Spark, PySpark, Python, and SQL with a mix of on-premise migration and modern Lakehouse architectures. This hybrid role requires in-office days in Chicago.

Qualifications

• 6+ years of hands-on Data Engineering experience.
• Strong Azure Databricks and Spark/PySpark expertise in production environments.
• Advanced Python and SQL development skills.
• Experience owning end-to-end data pipelines from source through consumption.
• Strong ETL/ELT, data modeling, and Lakehouse architecture experience.
• Experience with CI/CD, testing, monitoring, orchestration, and production support.

Responsibilities

• Design and build end-to-end data pipelines from ingestion through transformation and downstream consumption.
• Develop scalable ETL/ELT solutions supporting both batch and streaming/CDC workloads.
• Build solutions using Python, SQL, PySpark, Spark, and Azure Databricks.
• Implement Lakehouse architectures and Bronze/Silver/Gold data models.
• Migrate and modernize legacy warehouse environments into a centralized cloud platform.
• Optimize data models, schemas, performance, and scalability.
• Establish CI/CD, automated testing, monitoring, alerting, and operational best practices.
• Troubleshoot complex production data issues and support platform reliability.
• Participate in code reviews and mentor junior-level engineers.

Skills

Data engineering

Data pipelines

ETL/ELT

Data modeling

CI/CD

Tools

Azure Databricks

Spark

PySpark

Python

SQL

Job description

Location: Chicago, IL (Hybrid, 3+ Days Onsite)

Pay: $150K-$190K Base + Bonus

About the Role

A leading global professional services organization is undertaking a major enterprise-wide Data & AI modernization initiative and is investing heavily in a centralized cloud-based data ecosystem. This team is building the foundation for analytics, business intelligence, operational reporting, and future AI-driven capabilities.

This is an opportunity to join early in the journey and play a key role in shaping a modern Azure Databricks environment, building production-grade data pipelines, and helping establish engineering standards within a growing data organization.

Responsibilities

• Design and build end-to-end data pipelines from ingestion through transformation and downstream consumption.
• Develop scalable ETL/ELT solutions supporting both batch and streaming/CDC workloads.
• Build solutions using Python, SQL, PySpark, Spark, and Azure Databricks.
• Implement Lakehouse architectures and Bronze/Silver/Gold data models.
• Migrate and modernize legacy warehouse environments into a centralized cloud platform.
• Optimize data models, schemas, performance, and scalability.
• Establish CI/CD, automated testing, monitoring, alerting, and operational best practices.
• Troubleshoot complex production data issues and support platform reliability.
• Participate in code reviews and mentor junior-level engineers.

Must-Have Qualifications

• 6+ years of hands-on Data Engineering experience.
• Strong Azure Databricks and Spark/PySpark expertise in production environments.
• Advanced Python and SQL development skills.
• Experience owning end-to-end data pipelines from source through consumption.
• Strong ETL/ELT, data modeling, and Lakehouse architecture experience.
• Experience with CI/CD, testing, monitoring, orchestration, and production support.