Lead Databricks Lakehouse Architect (Remote)

Celebal Technologies; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Career development

Cutting-edge Databricks tech exposure

Global collaboration

Job summary

Celebal Technologies is seeking an experienced Databricks Architect to lead the design and implementation of scalable data lakehouse solutions on AWS and/or Azure. You will define the technical architecture, establish data governance, and drive best practices across BI and analytics teams.

The role requires deep expertise with Delta Live Tables, Medallion Architecture, and Unity Catalog, plus strong PySpark/SQL skills. Remote with travel as needed, full-time, US-ready.

Qualifications

• Strong hands-on experience in Databricks architecture and Lakehouse implementation.
• Extensive experience with AWS and/or Microsoft Azure cloud platforms.
• Strong expertise in Delta Live Tables (DLT) and pipeline development.
• Hands-on experience implementing Medallion Architecture (Bronze, Silver, Gold).
• Strong experience with Databricks Auto Loader for incremental and scalable data ingestion.
• Proficiency in PySpark, Python, and Spark SQL.
• Experience with Delta Lake, Unity Catalog, and data governance.
• Strong understanding of batch and real-time/streaming data processing.
• Experience with performance tuning, optimization, and large-scale data processing.
• Knowledge of CI/CD, Git, and DevOps practices for Databricks deployments.
• Excellent communication, stakeholder management, and technical leadership skills.

Responsibilities

• Design and implement scalable, secure, and high-performance Lakehouse architectures using Databricks.
• Define and implement end-to-end data engineering solutions on AWS and/or Azure.
• Architect and implement Medallion Architecture (Bronze, Silver, and Gold layers).
• Develop and optimize Delta Live Tables (DLT) pipelines for batch and streaming workloads.
• Implement Databricks Auto Loader for efficient and incremental ingestion of structured and unstructured data.
• Establish data governance, security, access controls, and best practices using Unity Catalog.
• Optimize Spark workloads, cluster configurations, job performance, and compute costs.
• Design and implement data integration and transformation frameworks using PySpark and SQL.
• Collaborate with business stakeholders, data engineers, and cross-functional teams to translate business requirements into technical solutions.
• Provide technical leadership, architecture guidance, code reviews, and mentorship to engineering teams.
• Ensure best practices around CI/CD, deployment automation, monitoring, and data quality.

Skills

Databricks architecture

Lakehouse implementation

AWS/Azure cloud

Delta Live Tables

Medallion Architecture

Databricks Auto Loader

PySpark / Spark SQL

Unity Catalog / data governance

CI/CD / DevOps for Databricks

Leadership / communication

Tools

Terraform

Git

Unity Catalog

Job description

Celebal Technologies is seeking an experienced Databricks Architect to lead the design and implementation of scalable data lakehouse solutions on AWS and/or Azure. You will define the technical architecture, establish data governance, and drive best practices across BI and analytics teams.

The role requires deep expertise with Delta Live Tables, Medallion Architecture, and Unity Catalog, plus strong PySpark/SQL skills. Remote with travel as needed, full-time, US-ready.