Lead Data Engineer / Databricks Solution Architect

Celebal Technologies

Dallas (TX)

On-site

USD 180,000 - 240,000

Full time

6 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Celebal Technologies is seeking a hands-on Lead Data Engineer and Databricks Solution Architect to design and scale an enterprise data platform on Databricks and AWS. You will transform operational data into trusted, AI-ready data products powering Tableau, Databricks Genie, ML, and self-service analytics across multiple business domains.

The role requires deep experience with AWS, PySpark, Azure Databricks, Delta Lake, and Unity Catalog, plus proficiency in SQL and Python.

Qualifications

  • 8+ years of experience with AWS.
  • 6+ years of experience with PySpark.
  • 6+ years of experience with Azure Databricks.
  • Strong expertise in Delta Lake, Medallion Architecture, and Unity Catalog.
  • Proficiency in SQL and Python.
  • Experience with data migration and ETL/ELT pipelines.

Responsibilities

  • Design and build reference architectures using Lakehouse and Delta Lake best practices.
  • Lead data migrations and the deployment of big data and AI applications on Databricks.
  • Enforce data security and governance with Unity Catalog.
  • Optimize clusters and pipeline performance; provide escalated production support.
  • Implement CI/CD pipelines and reusable assets for consistent delivery.
  • Train and enable customer teams through documentation and best practices.

Skills

SQL
Python
ETL/ELT

Tools

AWS
Azure Databricks
Delta Lake
Unity Catalog
Databricks Genie
CI/CD
Git
Infrastructure-as-code
PySpark

Job description

Locations: Grand Rapids, MI | Louisville, KY | Dallas, TX | Houston, TX | Chicago, IL | San Jose, CA

Employment Type: Contract / Full-Time

Certification: Databricks Certification preferred

Role Overview

Our client is seeking a hands‑on Lead Data Engineer and Databricks Solution Architect to build and scale an enterprise data platform on Databricks and AWS. You will turn operational data into trusted, AI‑ready data products that power Tableau, Databricks Genie, ML, and self‑service analytics across Manufacturing, Supply Chain, Sales, Finance, and HR.

Key Responsibilities
  • Design and build reference architectures using Lakehouse and Delta Lake best practices
  • Lead data migrations and the deployment of big data and AI applications on Databricks
  • Enforce data security and governance with Unity Catalog
  • Optimize clusters and pipeline performance; provide escalated production support
  • Implement CI/CD pipelines and reusable assets for consistent delivery
  • Train and enable customer teams through documentation and best practices.
Required Qualifications
  • 8+ years of experience with AWS
  • 6+ years of experience with PySpark
  • 6+ years of experience with Azure Databricks
  • Strong expertise in Delta Lake, Medallion Architecture, and Unity Catalog
  • Proficiency in SQL and Python
  • Experience with data migration and ETL/ELT pipelines
Preferred Qualifications
  • Databricks certification
  • Experience with Databricks Genie, Auto Loader, and Delta Live Tables
  • CI/CD, Git, and infrastructure‑as‑code experience

Email: Medha.shinde@celebaltech.com

#Hiring #LeadDataEngineer #DatabricksArchitect #Databricks #DataEngineering #AWS #PySpark #DeltaLake #UnityCatalog #HybridJobs #MichiganJobs #DallasJobs #HoustonJobs #ChicagoJobs #SanJoseJobs #LouisvilleJobs

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer (Databricks)
Lead Data Engineer (Databricks)

Celebal Technologies • Grand Rapids (MI)

Hybrid
USD 140,000 - 190,000
Team Lead / Solution Architect – (Databricks & Azure Migration)
Team Lead / Solution Architect – (Databricks & Azure Migration)

Celebal Technologies • Grand Rapids (MI)

On-site
USD 120,000 - 180,000
Databricks Data Architect
Databricks Data Architect

Winfo Solutions • Atlanta (GA)

On-site
USD 140,000 - 210,000
Lead Data Engineer
Lead Data Engineer

KDR Talent Solutions USA • United States

Remote
USD 198,000 - 242,000
Competitive compensation
Full benefits package
Flexible remote or hybrid options
+1
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

ACI Infotech • Atlanta (GA)

Hybrid
USD 100,000 - 130,000
Lead Data Engineer
Lead Data Engineer

Recru • Houston (TX)

On-site
USD 130,000 - 150,000
Lead Databricks Data Engineer
Lead Databricks Data Engineer

EXL Talent Acquisition Team • New York (NY)

On-site
USD 180,000 - 240,000
Databricks Solutions Architect
Databricks Solutions Architect

Compunnel Inc. • Florida

On-site
USD 120,000 - 160,000
Data Team Lead
Data Team Lead

Incedo Inc. • San Rafael (CA)

Hybrid
USD 130,000 - 150,000
Medical insurance
Vision insurance
401(k)
Lead Data Engineer
Lead Data Engineer

Globenet Consulting Corp • Bellevue (WA)

On-site
USD 140,000 - 180,000
Competitive salary
Opportunity for advancement
Training & development