Databricks Senior Data Engineer | Hybrid Chicago

Gulf Coast Automation Group

Chicago (IL)

Hybrid

USD 150,000 - 164,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

TalentFish is seeking a Senior Data Engineer to build a brand-new Azure Databricks platform in Chicago, IL. You will own end-to-end pipelines from ingestion to modeling on a greenfield data platform with executive sponsorship and wide visibility.

This hands-on role demands deep Python, SQL and Spark skills, Delta Lake and Unity Catalog experience, and the ability to raise the technical bar for a growing BI‑focused team.

Qualifications

  • 6+ years of hands-on data engineering delivering end-to-end pipelines.
  • Databricks experience (Azure preferred; AWS/GCP acceptable) including Delta Lake and cataloging.
  • Strong Python, SQL, and Apache Spark coding beyond scripting.
  • Design/build scalable ETL/ELT pipelines with reusable components.
  • Experience with object storage, lakehouse engines, orchestration, and streaming/CDC.
  • Solid data modeling, schema design, and performance tuning.
  • Git-based workflows, CI/CD, test automation, monitoring and alerting.
  • Ability to explain technical work and business impact clearly.

Responsibilities

  • Lead the design and build of scalable data pipelines and models on a brand-new Databricks platform.
  • Own pipelines end-to-end from ingestion to transformation for analytics, BI, and AI/ML uses.
  • Refactor and migrate legacy warehouses into the centralized Databricks lakehouse.
  • Code daily in Python, SQL, and Spark; use Databricks assets and API-driven jobs.
  • Establish CI/CD standards with GitHub/Azure DevOps and automated data testing.
  • Shape standards for data modeling and governance; ensure reliability and performance.
  • Mentor junior engineers; conduct code reviews and promote best practices.
  • Collaborate with data, BI, AI, finance, and product owners to translate impact.

Skills

Python
SQL
Apache Spark
ETL/ELT pipelines
Data modeling
CI/CD concepts
Troubleshooting

Education

Bachelor's degree in Computer Science, Engineering, Data Science, or related

Tools

Azure Databricks
Delta Lake
Unity Catalog
Databricks SQL
Spark clusters
GitHub
Azure DevOps
Terraform

Job description

TalentFish is seeking a Senior Data Engineer to build a brand-new Azure Databricks platform in Chicago, IL. You will own end-to-end pipelines from ingestion to modeling on a greenfield data platform with executive sponsorship and wide visibility.

This hands-on role demands deep Python, SQL and Spark skills, Delta Lake and Unity Catalog experience, and the ability to raise the technical bar for a growing BI‑focused team.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Data Engineer – Greenfield (Hybrid)
Senior Databricks Data Engineer – Greenfield (Hybrid)

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Health insurance
401(k)
Paid time off
Senior Data Engineer
Senior Data Engineer

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Senior Data Engineer (Databricks)
Senior Data Engineer (Databricks)

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Health insurance
401(k)
Paid time off
Senior Databricks Lakehouse Engineer - Azure Hybrid Chicago
Senior Databricks Lakehouse Engineer - Azure Hybrid Chicago

Bilink Corp • Chicago (IL)

Hybrid
USD 110,000 - 160,000
401(k) with match
Health and dental insurance
Paid time off
Senior Data Engineer - Databricks, Python & Azure
Senior Data Engineer - Databricks, Python & Azure

Motion Recruitment • Chicago (IL)

Remote
USD 120,000 - 180,000
Medical, Dental, and Vision Insurance
Vacation Time
Stock Options
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

ACI Infotech • Atlanta (GA)

Hybrid
USD 100,000 - 130,000
Data Engineer for Enterprise AI & Databricks
Data Engineer for Enterprise AI & Databricks

Focused • Chicago (IL)

On-site
USD 145,000 - 175,000
Databricks Engineer.
Databricks Engineer.

Healthcare Triangle, Inc. • Dallas (TX)

On-site
USD 120,000 - 160,000
AWS Databricks Engineer
AWS Databricks Engineer

Saransh Inc • Chicago (IL)

On-site
USD 140,000 - 200,000
Senior Data Engineer – Databricks, PySpark & CI/CD
Senior Data Engineer – Databricks, PySpark & CI/CD

84.51° • Chicago (IL)

On-site
USD 121,000 - 169,000
Health insurance
401(k) with Roth option
HSA matching
+2