Senior Databricks Data Engineer – Greenfield (Hybrid)

Gulf Coast Automation Group

Chicago (IL)

Hybrid

USD 150,000 - 164,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
401(k)
Paid time off

Job summary

TalentFish is seeking a Senior Data Engineer to build a brand-new Azure Databricks platform in Chicago, IL. You will own end-to-end data pipelines, design scalable models, and enforce governance standards on a hybrid schedule with on-site and remote work options.

You will write Python, SQL, and Spark code daily, mentor peers, and collaborate across data, BI, and finance teams to deliver fast, reliable data for analytics and AI use cases.

Qualifications

  • Bachelor's degree in Computer Science, Engineering, Data Science, or a related field.
  • 6+ years of hands-on data engineering experience with end-to-end pipelines (ingestion, validation, transformation, governance).
  • Databricks experience (Azure preferred; AWS/GCP acceptable) including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, Workflows/Jobs.
  • Strong coding skills in Python, SQL, and Apache Spark.
  • Proven experience designing and building scalable ETL/ELT pipelines; reusable, metadata-driven pipelines.
  • Experience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelines.
  • Solid data modeling, schema design, and performance tuning.
  • Git-based workflows, CI/CD, test automation, monitoring and alerting.
  • Troubleshooting skills; focus on scalability, reliability, and maintainability.
  • Ability to clearly explain what you built and its business impact.

Responsibilities

  • Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yet.
  • Own pipelines end to end from ingestion to transformation and outputs for analytics, BI, and AI/ML use cases.
  • Refactor and migrate legacy data warehouses into the centralized Databricks lakehouse.
  • Work daily in Python, SQL, and Spark with Databricks assets, workflows, and API-driven jobs.
  • Establish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring and alerting.
  • Shape and enforce standards for data modeling and governance.
  • Troubleshoot and optimize performance across the platform.
  • Mentor junior engineers, conduct code reviews, and lead by example in design and testing practices.
  • Partner with Data Engineering, BI, AI, Finance, and product owners to translate technical work into business impact.

Skills

Databricks (Azure)
Python
SQL
Apache Spark
Delta Lake
Unity Catalog
ETL/ELT pipelines
GitHub / Azure DevOps CI/CD
Data modeling
CI/CD tooling

Education

Bachelor's degree in Computer Science, Engineering, Data Science, or related field

Tools

Azure Databricks
GitHub

Job description

TalentFish is seeking a Senior Data Engineer to build a brand-new Azure Databricks platform in Chicago, IL. You will own end-to-end data pipelines, design scalable models, and enforce governance standards on a hybrid schedule with on-site and remote work options.

You will write Python, SQL, and Spark code daily, mentor peers, and collaborate across data, BI, and finance teams to deliver fast, reliable data for analytics and AI use cases.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer, Azure Databricks (Hybrid)
Senior Data Engineer, Azure Databricks (Hybrid)

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Health insurance
401(k)
Paid time off
Senior Data Engineer
Senior Data Engineer

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Health insurance
401(k)
Paid time off
Senior Data Engineer (Databricks)
Senior Data Engineer (Databricks)

Gulf Coast Automation Group • Chicago (IL)

Hybrid
USD 150,000 - 164,000
Health insurance
401(k)
Paid time off
Senior Data Engineer - Greenfield Databricks Lakehouse
Senior Data Engineer - Greenfield Databricks Lakehouse

Energy Jobline ZR • Mundelein (IL)

On-site
USD 120,000 - 180,000
Senior Databricks Lakehouse Engineer - Azure Hybrid Chicago
Senior Databricks Lakehouse Engineer - Azure Hybrid Chicago

Bilink Corp • Chicago (IL)

Hybrid
USD 110,000 - 160,000
401(k) with match
Health and dental insurance
Paid time off
Hybrid Senior Data Engineer, Finance & AI Pipelines
Hybrid Senior Data Engineer, Finance & AI Pipelines

Horizontal Talent • Chicago (IL)

Hybrid
USD 40,000 - 83,000
Senior Data Engineer - Databricks/PySpark/SQL | Remote
Senior Data Engineer - Databricks/PySpark/SQL | Remote

HumanIT Digital Consulting • United States

Remote
USD 28,000 - 37,000
15th month salary
Health insurance for family
Birthday off
+3
Senior Data Engineer - Databricks, Python & Azure
Senior Data Engineer - Databricks, Python & Azure

Motion Recruitment • Chicago (IL)

Remote
USD 120,000 - 180,000
Medical, Dental, and Vision Insurance
Vacation Time
Stock Options
Senior Databricks Data Engineer — Hybrid Indy
Senior Databricks Data Engineer — Hybrid Indy

SDG Group • Chicago (IL)

Hybrid
USD 140,000 - 190,000
Senior Databricks Data Architect - Hybrid Lakehouse Lead
Senior Databricks Data Architect - Hybrid Lakehouse Lead

NewVision • San Francisco (CA)

Hybrid
USD 140,000 - 190,000
Hybrid work model