Lead Data Engineer (Databricks)

Celebal Technologies

Grand Rapids (MI)

Hybrid

USD 140,000 - 190,000

Full time

2 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Celebal Technologies is seeking a Lead Data Engineer (Databricks) to architect, build, and operate scalable data lakehouse solutions on Databricks across Azure or AWS in a hybrid setup in Grand Rapids, MI.

This is a hands-on technical leadership role responsible for end-to-end platform design, end-to-end delivery, and guiding data engineering teams.

Key focus areas include Medallion Architecture, Delta Lake, and optimized Spark workloads, with Databricks certification preferred.

Qualifications

  • Databricks architecture and Lakehouse design experience.
  • Hands-on Medallion Architecture and Databricks Auto Loader.
  • Strong SQL, PySpark and Python skills.
  • Experience with Azure Databricks or AWS Databricks.
  • Knowledge of Spark, Delta Lake and ETL/ELT design.
  • Proven pipeline optimization and production-grade data engineering experience.
  • Knowledge of data governance, security and scalable architecture design.
  • Strong communication, analytical, and problem-solving skills.

Responsibilities

  • Design and implement enterprise-scale Databricks architectures using Medallion Architecture (Bronze, Silver, Gold).
  • Develop ETL/ELT pipelines using SQL, PySpark, and Python.
  • Build incremental and streaming ingestion pipelines with Databricks Auto Loader.
  • Implement Delta Lake solutions for reliable, ACID-compliant storage and processing.
  • Integrate Databricks with Azure or AWS cloud services.
  • Optimize Apache Spark workloads, SQL queries, cluster configurations, and pipeline performance.
  • Establish data governance, security, and access control standards.
  • Provide technical leadership and architectural guidance to data engineering teams.
  • Support production deployments, troubleshooting, and performance tuning.

Skills

Databricks architecture
Medallion Architecture
SQL
PySpark
Python
Azure Databricks
AWS Databricks
Spark
Delta Lake
ETL/ELT design
pipeline optimization
data governance
security
communication skills

Job description

Lead Data Engineer (Databricks) | Hybrid | Grand Rapids, MI

Location: Grand Rapids, MI (Hybrid)

Employment Type: Contract / Full-Time

Certification: Databricks Certification preferred

Role Overview

We are seeking a Lead Data Engineer to architect, build, and operate scalable, secure, high-performance data lakehouse solutions on Databricks across Azure or AWS. This is a hands-on technical leadership role responsible for end-to-end platform design and delivery.

Key Responsibilities
  • Design and implement enterprise-scale Databricks architectures using Medallion Architecture (Bronze, Silver, Gold)
  • Develop ETL/ELT pipelines using SQL, PySpark, and Python
  • Build incremental and streaming ingestion pipelines with Databricks Auto Loader
  • Implement Delta Lake solutions for reliable, ACID-compliant storage and processing
  • Integrate Databricks with Azure or AWS cloud services
  • Optimize Apache Spark workloads, SQL queries, cluster configurations, and pipeline performance
  • Establish data governance, security, and access control standards
  • Provide technical leadership and architectural guidance to data engineering teams
  • Support production deployments, troubleshooting, and performance tuning
Required Qualifications
  • Strong experience with Databricks architecture and Lakehouse solutions
  • Hands-on expertise in Medallion Architecture and Databricks Auto Loader
  • Proficiency in SQL, PySpark, and Python
  • Experience with Azure Databricks or AWS Databricks
  • Solid understanding of Apache Spark, Delta Lake, and ETL/ELT design
  • Proven experience in pipeline optimization and production-grade data engineering
  • Knowledge of data governance, security, and scalable architecture design
  • Strong communication, analytical, and problem-solving skills
Preferred Qualifications
  • Databricks certification
  • Experience with Unity Catalog and Delta Live Tables (Lakeflow Declarative Pipelines)
  • Experience with CI/CD, Git, and infrastructure-as-code tools
  • Background in cloud migration and enterprise data platform modernization
  • Experience leading technical teams or defining data architecture standards
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer / Databricks Solution Architect
Lead Data Engineer / Databricks Solution Architect

Celebal Technologies • Dallas (TX)

On-site
USD 180,000 - 240,000
Senior Databricks Data Engineer and Lakehouse Architect
Senior Databricks Data Engineer and Lakehouse Architect

Celebal Technologies • Grand Rapids (MI)

Hybrid
USD 140,000 - 190,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

ACI Infotech • Atlanta (GA)

Hybrid
USD 100,000 - 130,000
Team Lead / Solution Architect – (Databricks & Azure Migration)
Team Lead / Solution Architect – (Databricks & Azure Migration)

Celebal Technologies • Grand Rapids (MI)

On-site
USD 120,000 - 180,000
Lead Databricks Data Engineer
Lead Databricks Data Engineer

EXL Talent Acquisition Team • New York (NY)

On-site
USD 180,000 - 240,000
Lead Databricks Engineer
Lead Databricks Engineer

Programmers • Minneapolis (MN)

On-site
USD 120,000 - 170,000
Databricks Data Engineer
Databricks Data Engineer

Compunnel Inc. • Spring (TX)

On-site
USD 110,000 - 140,000
Lead Data Engineer (Lakehouse / Databricks)
Lead Data Engineer (Lakehouse / Databricks)

Stafford Gray • Lansing (MI)

Hybrid
USD 140,000 - 180,000
Lead Data Engineer (Lakehouse / Databricks)
Lead Data Engineer (Lakehouse / Databricks)

Stafford Gray • Town of Lansing (NY)

Hybrid
USD 130,000 - 180,000
Lead Data Engineer – Databricks
Lead Data Engineer – Databricks

TechDigital Group • Bolingbrook (IL)

On-site
USD 120,000 - 150,000