Lead Data Engineer & Databricks Architect for AI-Ready Data

Celebal Technologies

Dallas (TX)

On-site

USD 180,000 - 240,000

Full time

26 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Celebal Technologies is seeking a hands-on Lead Data Engineer and Databricks Solution Architect to design and scale an enterprise data platform on Databricks and AWS. You will transform operational data into trusted, AI-ready data products powering Tableau, Databricks Genie, ML, and self-service analytics across multiple business domains.

The role requires deep experience with AWS, PySpark, Azure Databricks, Delta Lake, and Unity Catalog, plus proficiency in SQL and Python.

Qualifications

  • 8+ years of experience with AWS.
  • 6+ years of experience with PySpark.
  • 6+ years of experience with Azure Databricks.
  • Strong expertise in Delta Lake, Medallion Architecture, and Unity Catalog.
  • Proficiency in SQL and Python.
  • Experience with data migration and ETL/ELT pipelines.

Responsibilities

  • Design and build reference architectures using Lakehouse and Delta Lake best practices.
  • Lead data migrations and the deployment of big data and AI applications on Databricks.
  • Enforce data security and governance with Unity Catalog.
  • Optimize clusters and pipeline performance; provide escalated production support.
  • Implement CI/CD pipelines and reusable assets for consistent delivery.
  • Train and enable customer teams through documentation and best practices.

Skills

SQL
Python
ETL/ELT

Tools

AWS
Azure Databricks
Delta Lake
Unity Catalog
Databricks Genie
CI/CD
Git
Infrastructure-as-code
PySpark

Job description

Celebal Technologies is seeking a hands-on Lead Data Engineer and Databricks Solution Architect to design and scale an enterprise data platform on Databricks and AWS. You will transform operational data into trusted, AI-ready data products powering Tableau, Databricks Genie, ML, and self-service analytics across multiple business domains.

The role requires deep experience with AWS, PySpark, Azure Databricks, Delta Lake, and Unity Catalog, plus proficiency in SQL and Python.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Platform Architect – AI-Ready Data
Senior Databricks Platform Architect – AI-Ready Data

Celebal Technologies • Dallas (TX)

On-site
USD 140,000 - 190,000
Lead Data Architect & Engineer - Databricks Lakehouse
Lead Data Architect & Engineer - Databricks Lakehouse

CloudBigD • Atlanta (GA)

On-site
USD 89,000 - 126,000
Lead Data & AI Solutions Architect
Lead Data & AI Solutions Architect

WinsAbove • San Francisco (CA)

On-site
USD 180,000 - 248,000
Senior Databricks Solution Architect (Hands-On)
Senior Databricks Solution Architect (Hands-On)

Celebal Technologies • Houston (TX)

On-site
USD 140,000 - 180,000
Lead Data Engineer / Databricks Solution Architect
Lead Data Engineer / Databricks Solution Architect

Celebal Technologies • Dallas (TX)

On-site
USD 180,000 - 240,000
Data Engineer for AI Data Platform (Databricks)
Data Engineer for AI Data Platform (Databricks)

Focused • Denver (CO)

Hybrid
USD 140,000 - 170,000
Databricks & AI Consultant
Databricks & AI Consultant

Winfo Solutions • Texas

On-site
USD 110,000 - 165,000
Lead Data Engineer – Databricks
Lead Data Engineer – Databricks

TechDigital Group • Bolingbrook (IL)

On-site
USD 120,000 - 150,000
Lead Data & AI Solutions Architect
Lead Data & AI Solutions Architect

Neon • Massachusetts

On-site
USD 180,000 - 248,000
Lead Databricks Lakehouse Architect (Remote)
Lead Databricks Lakehouse Architect (Remote)

Celebal Technologies • United States

Remote
USD 140,000 - 210,000
Career development
Cutting-edge Databricks tech exposure
Global collaboration