RSA Databricks Architect

Celebal Technologies; Hyderabad, Bengaluru; On-site; INR 3,600,000 - 9,000,000; Full time

Job summary

Celebal Technologies is seeking a Senior Architect Databricks to lead design and delivery of scalable Lakehouse solutions for enterprise customers. You will guide data modernization programs, migrations from Teradata, Oracle, Netezza, SAS, and Snowflake, and mentor teams on best practices across Azure, AWS, or GCP environments.

You will work closely with CDOs, Data Architects, and Engineering Leaders, delivering end-to-end architectures and engaging in executive-level discussions while staying

Qualifications

• 12+ years of experience in Data Engineering, Data Platform Architecture, or Solution Architecture.
• Minimum 8+ years in Data Engineering/Data Platform Architecture with at least 3+ years of hands-on Databricks experience in a consulting or customer-facing role.
• Experience leading multiple enterprise-scale EDW modernization or Lakehouse migration programs.
• Strong expertise in Databricks Data Intelligence Platform and Lakehouse Architecture.
• Hands-on experience with Delta Lake, Apache Spark and PySpark, Unity Catalog, Lakeflow, Serverless Compute, Databricks SQL, AI/BI Dashboards, MLflow, Mosaic AI, DBT, and Databricks Auto Loader.
• Strong SQL and PySpark programming skills.
• Expertise in Medallion Architecture, Dimensional Modeling, and Data Governance.
• Experience with cloud platforms such as Azure, AWS, or GCP, including networking, IAM, and storage services.

Responsibilities

• Lead customer discussions from discovery, solution design, and architecture through implementation.
• Design end-to-end Databricks Lakehouse architectures covering ingestion, transformation, governance, machine learning, and analytics.
• Drive enterprise EDW modernization and migration programs from platforms such as Teradata, Oracle, Netezza, SAS, Greenplum, Hadoop, Synapse, and Snowflake to Databricks.
• Define migration strategies, wave planning, risk mitigation, and ROI/TCO assessments.
• Implement and recommend advanced Databricks capabilities, including Lakeflow Declarative Pipelines, Lakeflow Connect, Serverless Compute, Unity Catalog, Databricks Asset Bundles, Databricks SQL, AI/BI Dashboards, Genie, MLflow, Model Serving, Mosaic AI, DBT, and Databricks Auto Loader.
• Build production-grade ETL pipelines using DBT and Databricks Auto Loader.
• Conduct architecture workshops, PoCs, executive presentations, and technical deep dives.
• Prepare architecture documentation, including HLDs, LLDs, migration roadmaps, reference architectures, and runbooks.
• Collaborate with Sales, Delivery, and Databricks teams to support proposals, solutioning, and SoWs.
• Stay current with the Databricks product roadmap and promote new capabilities internally and externally.
• Mentor architects and data engineers on Databricks best practices.

Skills

Stakeholder mgmt

Leadership

Mentoring

Solution design

EDW modernization

Lakehouse architecture

Cloud platforms

SQL

PySpark

Databricks

Tools

Delta Lake

Apache Spark

PySpark

Lakeflow Declarative Pipelines

Lakeflow Connect

Serverless Compute

Unity Catalog

Databricks SQL

MLflow

Databricks Auto Loader

DBT

Job description

About the RoleCelebal Technologies is looking for an experiencedSenior Architect Databricksto join our growing Databricks Practice. In this customer-facing consulting role, you will work closely with enterprise customers, including CDOs, Data Architects, and Engineering Leaders, to design and deliver scalable Lakehouse solutions on the Databricks Data Intelligence Platform.You will lead enterprise data modernization initiatives, drive large-scale EDW migrations, and provide architectural guidance while leveraging the latest Databricks capabilities, including Lakeflow, Unity Catalog, Serverless Compute, MLflow, and AI/BI solutions.Key Responsibilities

• Lead customer discussions from discovery, solution design, and architecture through implementation.
• Design end-to-end Databricks Lakehouse architectures covering ingestion, transformation, governance, machine learning, and analytics.
• Drive enterprise EDW modernization and migration programs from platforms such as Teradata, Oracle, Netezza, SAS, Greenplum, Hadoop, Synapse, and Snowflake to Databricks.
• Define migration strategies, wave planning, risk mitigation, and ROI/TCO assessments.
• Implement and recommend advanced Databricks capabilities, including:
• Lakeflow Declarative Pipelines (LDP)
• Lakeflow Connect
• Serverless Compute
• Unity Catalog
• Databricks Asset Bundles (DAB)
• Databricks SQL, AI/BI Dashboards, and Genie
• MLflow, Model Serving, and Mosaic AI
• Build production-grade ETL pipelines using DBT and Databricks Auto Loader.
• Conduct architecture workshops, PoCs, executive presentations, and technical deep dives.
• Prepare architecture documentation, including HLDs, LLDs, migration roadmaps, reference architectures, and runbooks.
• Collaborate with Sales, Delivery, and Databricks teams to support proposals, solutioning, and Statements of Work (SoWs).
• Stay current with the Databricks product roadmap and promote new capabilities internally and externally.
• Mentor architects and data engineers on Databricks best practices.

Required Experience

• 12+ years of experience in Data Engineering, Data Platform Architecture, or Solution Architecture.
• Minimum 8+ years in Data Engineering/Data Platform Architecture with at least 3+ years of hands‑on Databricks experience in a consulting or customer‑facing role.
• Experience leading multiple enterprise‑scale EDW modernization or Lakehouse migration programs.
• Strong expertise in Databricks Data Intelligence Platform and Lakehouse Architecture.
• Hands‑on experience with:
• Delta Lake
• Apache Spark and PySpark
• Unity Catalog
• Lakeflow Declarative Pipelines
• Lakeflow Connect
• Serverless Compute
• Databricks SQL
• AI/BI Dashboards
• MLflow
• Mosaic AI
• DBT
• Databricks Auto Loader
• Strong SQL and PySpark programming skills.
• Expertise in Medallion Architecture, Dimensional Modeling, and Data Governance.
• Experience with cloud platforms such as Azure, AWS, or GCP, including networking, IAM, and storage services (ADLS Gen2, Amazon S3, or Google Cloud Storage).
• Excellent stakeholder management and communication skills with experience engaging technical teams and C‑level executives.
• Willingness to travel internationally for customer engagements when required.

Preferred Skills

• Disaster Recovery architecture (Delta Deep Clone, cross‑region replication, RPO/RTO design)
• MLOps and Generative AI implementation using MLflow, Model Serving, Vector Search, and Mosaic AI
• Experience with Informatica, ODI, SAS DI, IBM DataStage, or Talend
• Infrastructure as Code using Terraform and Databricks Asset Bundles
• Governance tools such as Immuta, Collibra, Microsoft Purview, or Alation
• Domain experience in BFSI, Insurance, Retail, Manufacturing, or Healthcare

What We're Looking For

• Proven experience owning solution architecture from discovery through production deployment.
• Ability to act as a trusted advisor for enterprise customers adopting Databricks.
• Strong leadership and mentoring capabilities.
• Experience building reusable accelerators, reference architectures, and migration frameworks.
• Strong collaboration with Databricks field teams on strategic customer engagements.