Senior Data Engineer

Harnham; Chicago (IL); Hybrid; USD 150,000 - 210,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Confidential Law Firm is building a next-generation enterprise data platform in Chicago and seeks a Senior Data Engineer to own production pipelines end-to-end. You will shape architecture, implement ETL/ELT and streaming workloads, and advance Delta Lake/ Lakehouse patterns across a global organization.

The role emphasizes hands-on coding, strong Python/SQL skills, and modern CI/CD, with Azure as the preferred cloud.

Qualifications

• 6+ years of substantive, hands-on Data Engineering experience.
• Deep production expertise with Databricks and Spark/PySpark.
• Strong Python + SQL fundamentals and data modeling.
• Experience with CI/CD, testing, monitoring and performance optimization.
• Azure cloud experience preferred; greenfield project ownership.

Responsibilities

• Architect and build production-grade data pipelines end-to-end.
• Engineer ETL/ELT, batch and streaming workloads.
• Work with Azure Databricks, Python, SQL, PySpark and Delta Lake.
• Define data models and patterns (Bronze/Silver/Gold).
• Modernize legacy warehouses and own CI/CD, testing, and observability.
• mentor engineers and influence technical direction while coding hands-on.

Skills

Python

SQL

Data modeling

CI/CD

Observability

Education

Bachelor's degree in CS/CE/Math/Data Science

Tools

Databricks

Spark/PySpark

Delta Lake

Job description

Senior Data Engineer | Greenfield Data Platform & AI Transformation | Chicago

I’m partnering with a globally renowned law firm making a significant investment in Data & AI in its sector. We’re searching for an exceptional Senior Data Engineer to join the foundational team building its next-generation enterprise data platform. This is not a maintenance role.

The organization has selected Azure Databricks as its strategic data platform, but the production engineering ecosystem is still being built. You’ll join at a rare inflection point: enterprise-scale resources and executive sponsorship, with the technical ownership and greenfield opportunity of an early-stage platform team.

The Challenge

You’ll help establish the engineering foundation that will power analytics, data products and increasingly sophisticated AI capabilities across a complex global organization.

You will:

• Architect and build production-grade pipelines end-to-end, from ingestion and validation through transformation, modeling and consumption.


• Engineer scalable ETL/ELT, batch and streaming/CDC workloads.


• Build deeply with Azure Databricks, Python, SQL, PySpark/Spark and Delta Lake/Lakehouse architectures.


• Help establish Bronze/Silver/Gold patterns, data models and engineering standards.


• Modernize and migrate legacy warehouse environments into Databricks.


• Own CI/CD, automated testing, observability, monitoring and production reliability.


• Diagnose complex performance problems across Spark workloads, joins, shuffles, partitioning and distributed pipelines.


• Influence architecture and technical direction while remaining deeply hands-on in the code.


• Review code, mentor engineers and help define what excellent Data Engineering looks like as the organization scales.

The Bar Is Intentionally High

We are looking for a top-tier engineer and builder, not someone whose career has moved primarily into management, BI/reporting or high-level architecture.

The strongest candidates will bring:

• 6+ years of substantive, hands-on Data Engineering experience.


• Deep production expertise with Databricks and Spark/PySpark.


• Exceptional Python + SQL fundamentals.


• Strong Lakehouse, Delta Lake, data modeling and distributed-processing fundamentals.


• Experience building engineering rigor around CI/CD, testing, monitoring, orchestration and performance optimization.


• Modern cloud experience, with Azure strongly preferred.


• Evidence of building or materially transforming data platforms rather than simply operating within them.


• Strong technical judgment, intellectual horsepower and the communication skills to operate with sophisticated senior stakeholders.


• A strong U.S. university academic foundation in Computer Science, Computer Engineering, Engineering, Mathematics, Statistics, Data Science or another rigorous quantitative discipline is highly preferred.

We are particularly interested in engineers coming from elite technology companies, sophisticated legal technology organizations, leading fintech/financial institutions, or other highly regarded engineering environments.

Location & Work Authorization

This opportunity is Chicago-based and hybrid, with three days onsite per week.

Candidates must already possess unrestricted U.S. work authorization without current or future employer sponsorship (e.g., U.S. Citizen or Green Card holder).

Why Consider It?

If you already work in a mature engineering organization, the differentiator here is ownership.

You are not joining after every important architectural decision has been made. You’re joining while a globally recognized institution is building the platform, standards and engineering organization that could define its Data & AI capabilities for the next decade.

It offers an unusual combination of institutional prestige, stability and resources with greenfield technical ownership, executive visibility and genuine influence.

Competitive compensation + bonus and excellent enterprise benefits.

If you are an exceptional hands-on Sr. Data Engineer who wants your next role to be defined by what you build and influence, rather than what you inherit,