Databricks Data Architect/Lead

Celebal Technologies; United States; On-site; USD 180,000 - 240,000; Full time

Job summary

Celebal Technologies seeks a Senior Solution Architect for our Databricks Practice to shape enterprise data modernization journeys. This role designs and delivers Lakehouse solutions on the Databricks Data Intelligence Platform, collaborating with CDOs, Data Architects, and Engineering leaders.

You will translate business problems into architectural blueprints, guide EDW modernization, and drive adoption of Lakeflow, Unity Catalog, and serverless compute.

Qualifications

• 8+ years in data engineering / data platform architecture.
• 3+ years hands-on with Databricks in a customer-facing or consulting capacity.
• Led at least 2–3 large EDW modernization or Lakehouse migration programs end to end.
• Strong understanding of the Databricks Data Intelligence Platform.
• Delta Lake internals and optimization techniques.
• Spark execution model, photon, cluster sizing and performance tuning.
• Unity Catalog object model (metastore, catalogs, schemas, volumes, external locations, storage credentials).
• Multi-cloud experience across AWS / Azure / GCP.
• Hands-on with Lakeflow Declarative Pipelines, Lakeflow Connect, and Serverless workloads.
• Medallion Architecture, dimensional modeling, data governance frameworks.
• Strong SQL and PySpark; ability to read Spark plans.
• Cloud fluency on Azure/AWS/GCP including networking, IAM, and storage layers.
• Excellent communication and stakeholder management with engineers and C-suite.
• Willing to travel and stay overseas for solutioning and delivery projects.

Responsibilities

• Lead architecture for EDW modernization or Lakehouse migration programs end to end.
• Be the go-to advisor on advanced Databricks adoption (LDP, Lakeflow) within the practice.
• Contribute to reusable accelerators, reference architectures, and migration playbooks.
• Build strong relationships with Databricks field teams and support joint pursuits.

Skills

SQL & PySpark

Stakeholder communication

Travel willingness

Tools

Databricks platform

Terraform

Immuta

Collibra

Purview

Alation

MLflow

Informatica

Talend

DataStage

Job description

Experience Required: 8-13 Years (Min 4+ years hands-on with Databricks)

About the Role

Job Summary:

We are seeking a Senior Solution Architect to join our Databricks Practice and play a pivotal role in shaping data modernization journeys for enterprise customers. This is a customer-facing, consulting-oriented role where you will engage with CDOs, Data Architects, and Engineering leaders to design and deliver scalable, future-ready Lakehouse solutions on the Databricks Data Intelligence Platform.

You will be the trusted technical advisor — translating business problems into architectural blueprints, guiding customers through EDW modernization programs, and championing the adoption of Databricks' most advanced capabilities including Lakeflow Declarative Pipelines (LDP), Lakeflow Connect, Serverless Compute, Unity Catalog, and AI/BI workloads.

Qualifications

• 8+ years in data engineering / data platform architecture, with 3+ years handson on Databricks in a customer-facing or consulting capacity.
• Demonstrated experience leading at least 2–3 large- scale EDW modernization or Lakehouse migration programs end to end.
• Strong, fundamentals-level understanding of the Databricks Data Intelligence Platform.
• Delta Lake internals (transaction log, OPTIMIZE, Z- order, Liquid Clustering, Deletion Vector, Predictive Optimization)
• Spark execution model, Photon, cluster sizing, and performance tuning.
• Unity Catalog object model (metastore, catalogs, schemas, volumes, external locations, stor age credentials).
• Workspace, account, and identity architecture across AWS / Azure / GCP.
• Handson experience designing and deploying Lakeflow Declarative Pipelines, Lakeflow Connect, and Serverless workloads in production.
• Solid grounding in Medallion Architecture, dimensional modeling, and data governance frameworks.
• Strong SQL and PySpark skills; comfort reading and reasoning about Spark execution plans.
• Cloud fluency on at least one of Azure, AWS, or GCP including networking, IAM, and storage layers (ADLS Gen2 / S3 / GCS).
• Excellent communication and stakeholder management skills able to engage equally with engineers and C- suite.
• Willing to travel and stay overseas for Solutioning and Delivering projects.

Required Skills

• Databricks certifications: Certified Data Engineer Professional, Certified Machine Learning Professional, or Databricks Certified Solutions Architect Professional.
• Experience with DR architectures (Delta Deep Clone, cross region replication, RPO/RTO design).
• Exposure to MLOps / GenAI patterns on Databricks (MLflow, Model Serving, Vector Search, Mosaic AI Agent Framework)
• Familiarity with legacy ETL/ELT platforms (Informatica, ODI, SAS DI, DataStage, Talend) for migration credibility.
• Experience with Terraform / Databricks Asset Bundles for IaC and CI/CD.
• Working knowledge of Immuta, Collibra, Purview, or Alation for governance integration.
• Industry depth in BFSI, Insurance, Retail, Manufacturing, or Healthcare.

Preferred Skills

• Owned the architecture for at least 2 strategic customer engagements from discovery to go live.
• Established yourself as the goto advisor on advanced Databricks adoption (LDP, Lakeflow Connect, Serverless) within the practice
• Contributed to reusable accelerators, reference architectures, and migration playbooks that scale across the practice.
• Built strong working relationships with Databricks field teams and contributed to joint pursuits.