Senior Data Engineer

Gulf Coast Automation Group

Chicago (IL)

Híbrido

USD 150 000 - 164 000

Tempo integral

Há 3 dias
Torna-te num dos primeiros candidatos

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Resumo da oferta

TalentFish is seeking a Senior Data Engineer to build a brand-new Azure Databricks platform in Chicago, IL. You will own end-to-end pipelines from ingestion to modeling on a greenfield data platform with executive sponsorship and wide visibility.

This hands-on role demands deep Python, SQL and Spark skills, Delta Lake and Unity Catalog experience, and the ability to raise the technical bar for a growing BI‑focused team.

Qualificações

  • 6+ years of hands-on data engineering delivering end-to-end pipelines.
  • Databricks experience (Azure preferred; AWS/GCP acceptable) including Delta Lake and cataloging.
  • Strong Python, SQL, and Apache Spark coding beyond scripting.
  • Design/build scalable ETL/ELT pipelines with reusable components.
  • Experience with object storage, lakehouse engines, orchestration, and streaming/CDC.
  • Solid data modeling, schema design, and performance tuning.
  • Git-based workflows, CI/CD, test automation, monitoring and alerting.
  • Ability to explain technical work and business impact clearly.

Responsabilidades

  • Lead the design and build of scalable data pipelines and models on a brand-new Databricks platform.
  • Own pipelines end-to-end from ingestion to transformation for analytics, BI, and AI/ML uses.
  • Refactor and migrate legacy warehouses into the centralized Databricks lakehouse.
  • Code daily in Python, SQL, and Spark; use Databricks assets and API-driven jobs.
  • Establish CI/CD standards with GitHub/Azure DevOps and automated data testing.
  • Shape standards for data modeling and governance; ensure reliability and performance.
  • Mentor junior engineers; conduct code reviews and promote best practices.
  • Collaborate with data, BI, AI, finance, and product owners to translate impact.

Conhecimentos

Python
SQL
Apache Spark
ETL/ELT pipelines
Data modeling
CI/CD concepts
Troubleshooting

Formação académica

Bachelor's degree in Computer Science, Engineering, Data Science, or related

Ferramentas

Azure Databricks
Delta Lake
Unity Catalog
Databricks SQL
Spark clusters
GitHub
Azure DevOps
Terraform

Descrição da oferta de emprego

Job Title: Senior Data Engineer (Databricks)
Primary Location: Chicago, IL (Hybrid – 3 days onsite / 2 days remote)
Position Type: Full-Time, Direct Hire

OVERVIEW

TalentFish is casting a line for a Senior Data Engineer. This is a full-time, direct hire role in Chicago, IL, working a hybrid schedule with a minimum of three days onsite per week. This role exists as part of a multi-year initiative to transform our client's Data & AI function, building a modern data platform that enables speed-to-insight and makes the organization truly data-driven.
You'll be one of the core builders of a brand-new Azure Databricks platform, designing and delivering the pipelines, transformations, and models that everything else runs on. The environment is greenfield: Databricks is stood up, but no production pipelines exist yet, so you'll help set the standards from day one. This is a hands‑on senior individual contributor role, not a people management, architecture, AI/LLM, or BI/reporting role. The architecture is defined; your job is to build it well and raise the technical bar of a growing team (expanding to 25–30) that is largely BI-focused today.
The work carries strong executive sponsorship and enterprise-wide visibility, with the chance to influence engineering practices across the organization. The interview process includes a live technical assessment covering Python, SQL, and practical data pipeline logic.

WHAT YOU BRING TO THE ROLE. (IDEAL EXPERIENCE)
  • Bachelor's degree in Computer Science, Engineering, Data Science, or a related field
  • 6+ years of hands‑on data engineering experience, with a track record of owning pipelines end to end: ingestion, validation, transformation, and governance
  • Real, hands‑on Databricks experience (Azure preferred; AWS or GCP acceptable), including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, and Workflows/Jobs
  • Strong coding skills in Python, SQL, and Apache Spark that go well beyond basic scripting
  • Proven experience designing and building scalable ETL/ELT pipelines, including reusable, metadata‑driven pipelines
  • Experience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelines
  • Solid data modeling, schema design, and performance tuning experience
  • Working knowledge of Git‑based workflows, CI/CD, test automation, and monitoring and alerting
  • Strong troubleshooting skills and a solid grasp of engineering fundamentals: scalability, reliability, and maintainability
  • The ability to clearly explain what you built, why it mattered, and how it solved a business or platform problem
Preferred
  • Experience building or migrating a data platform from the ground up
  • Infrastructure as Code (Terraform, Bicep, ARM) and Governance as Code experience
  • Multi‑cloud background with depth in Azure
  • Legacy data warehousing and/or Hadoop experience
  • Deep Databricks exposure beyond basic pipelines, such as enterprise‑scale work and optimization
  • A track record of mentoring and upskilling peers
  • Agile delivery experience
WHAT YOU'LL DO. (SKILLS USED IN THIS POSITION)
  • Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yet
  • Own pipelines end to end, from ingesting source systems to transforming and modeling data to producing outputs for downstream analytics, BI, and AI/ML use cases
  • Refactor and migrate multiple legacy data warehouses into the centralized Databricks lakehouse
  • Work at the code level every day in Python, SQL, and Spark, building pipelines with Databricks Asset Bundles, Spark workflows, and API‑driven jobs
  • Establish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring, and alerting
  • Help shape and enforce standards for data modeling and governance
  • Troubleshoot and optimize performance across the platform
  • Mentor and coach junior engineers, conduct code reviews, and lead by example in design, coding, and testing practices
  • Partner with Data Engineering, BI, AI, Finance, and product owners, and help translate technical work into business impact
COMPENSATION INFORMATION

The expected salary range for this position i s $150,000-$164,000 per year, depending on experience and qualifications. This role also qualifies for comprehensive benefits such as health insurance, 401(k), and paid time off. TalentFish is committed to pay transparency and equal opportunity. The salary range provided is in compliance with applicable state and federal regulations.
This role requires authorization to work in the U.S. without current or future visa sponsorship.
All offers are contingent upon the completion of a background check, which may include but is not limited to reference checks, education verification, employment verification, drug testing, criminal records checks, and any required certifications or compliance requirements based on the end client's background check policies and applicable laws.
TalentFish is an employee‑owned company pioneering a new realm in talent acquisition. We are redefining IT staffing by evolving AI, video screening, and our unique platform. TalentFish focuses on providing the best employee, consultant, and client experience possible. At TalentFish we are an Equal Opportunity Employer; we embrace and encourage diversity!

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Senior Data Engineer (Databricks)
Senior Data Engineer (Databricks)

Gulf Coast Automation Group • Chicago (IL)

Híbrido
USD 150 000 - 164 000
Health insurance
401(k)
Paid time off
Senior Data Engineer
Senior Data Engineer

Burtch Works • Chicago (IL)

Híbrido
USD 148 000 - 164 000
Senior Data Architect and Engineer
Senior Data Architect and Engineer

Gulf Coast Automation Group • Chicago (IL)

Híbrido
USD 165 000 - 220 000
Senior Databricks Data Engineer
Senior Databricks Data Engineer

CGI Group, Inc. • Miami (FL)

Híbrido
USD 92 000 - 204 000
Competitive compensation
Comprehensive insurance options
401(k) match and share purchase plan
+3
Databricks Senior Data Engineer | Hybrid Chicago
Databricks Senior Data Engineer | Hybrid Chicago

Gulf Coast Automation Group • Chicago (IL)

Híbrido
USD 150 000 - 164 000
Specialist Solutions Architect - Data Engineering & Warehousing (Financial Services)
Specialist Solutions Architect - Data Engineering & Warehousing (Financial Services)

WinsAbove • EUA

Híbrido
USD 149 000 - 248 000
Benefits
Senior Data & AI Solutions Architect
Senior Data & AI Solutions Architect

Neon • California

Presencial
USD 182 000 - 250 000
Sr. Solutions Architect - AI Natives Business
Sr. Solutions Architect - AI Natives Business

Databricks Inc. • Oregon

Híbrido
USD 219 000 - 301 000
Remote work
Competitive compensation
Pay transparency
Senior Staff Software Engineer - Lakeflow Pipelines Datasets
Senior Staff Software Engineer - Lakeflow Pipelines Datasets

Databricks Inc. • San Francisco (CA)

Híbrido
USD 228 600 - 314 250
Comprehensive benefits
Diversity and inclusion commitment
Senior Data Engineer (Databricks) - Contractor
Senior Data Engineer (Databricks) - Contractor

Jobgether • EUA

Teletrabalho
USD 96 000 - 165 000
Remote work
Contract 6 months
Extension possibility