Databricks Architect

Idexcel; Bengaluru; On-site; INR 4,500,000 - 7,500,000; Full time

Job summary

Idexcel seeks a Databricks Architect with 10+ years of IT experience and 5+ years hands-on Databricks to design and deliver enterprise-scale Data, Analytics, AI, and Lakehouse solutions for customers.

You will lead architecture initiatives, build PoCs, mentor teams, and engage with stakeholders to drive cloud modernization and governance across cloud platforms.

Qualifications

• 10+ years of IT experience with strong data/analytics focus.
• 5+ years hands-on Databricks experience.
• Experience architecting enterprise-scale data lakehouse solutions.
• Experience with cloud modernization and customer-facing engagements.

Responsibilities

• Architect and implement enterprise Data Lakehouse solutions with Databricks.
• Design batch, streaming, real-time data pipelines.
• Build PoCs, accelerators, and reference architectures.
• Lead Databricks Genie implementations including governance and security.
• Tune Spark/Databricks performance and cost optimization.
• Drive data governance, Unity Catalog adoption, and platform observability.
• Lead data modernization, cloud migrations, and presales support.

Skills

PySpark

Spark SQL

Python

SQL

Data Modeling

Data Warehousing

Kafka

CDC architectures

AWS / Azure / GCP familiarity

Education

Bachelor's/Master's in CS/Engineering/IT/Data Science

Tools

Databricks Lakehouse Platform

Delta Lake

Unity Catalog

Lakeflow

Delta Live Tables (DLT)

Databricks Workflows

MLflow

Genie

Vector Search

Databricks Asset Bundles (DAB)

GitHub Actions

Terraform

Databricks CLI

Infrastructure as Code (IaC)

Power BI

Tableau

Superset

Job description

Databricks Architect (10+ Years Experience | 5+ Years Databricks)

Position Summary

Seeking a highly experienced Databricks Architect with 10+ years of IT experience and 5+ years of hands-on Databricks expertise to architect, design, and deliver enterprise-scale Data, Analytics, AI, and Lakehouse solutions. The candidate will lead architecture initiatives, build PoCs, drive cloud modernization, mentor engineering teams, and engage directly with customers and business stakeholders.

Key Responsibilities

• Architect and implement enterprise Data Lakehouse solutions using Databricks, Delta Lake, Unity Catalog, Lakeflow, Delta Live Tables (DLT), Workflows, MLflow, Mosaic AI, Genie, Vector Search, and AI/BI Dashboards.
• Design batch, streaming, CDC, real-time, and Medallion Architecture data pipelines.
• Build Proof of Concepts (PoCs), accelerators, reusable frameworks, reference architectures, and conduct customer demonstrations.
• Lead Databricks Genie implementations including conversational analytics, prompt engineering, governance, security, performance optimization, and enterprise integrations.
• Perform Spark and Databricks performance tuning, query optimization, workload optimization, AQE, partitioning, clustering, Liquid Clustering, caching, Photon optimization, and cost optimization.
• Drive data governance, security, Unity Catalog adoption, data lineage, and platform observability.
• Lead data modernization, cloud migration, architecture reviews, estimation, presales support, and stakeholder management.

Required Skills

• Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Lakeflow, DLT, Databricks Workflows, Databricks Asset Bundles (DAB), MLflow, Genie, Vector Search.
• Expert knowledge of PySpark, Spark SQL, Python, SQL, Data Modeling, Data Warehousing, Kafka, and CDC architectures.
• Experience with AWS, Azure, or GCP cloud platforms.
• Strong understanding of Databricks security, multi-tenant architectures, governance, and FinOps.
• Experience integrating Databricks with PostgreSQL, Snowflake, SQL Server, Power BI, Tableau, Superset, REST APIs, Salesforce, and SAP.
• Hands-on experience with Databricks Asset Bundles (DAB), GitHub Actions, Terraform, Databricks CLI, and Infrastructure as Code (IaC).
• Design CI/CD pipelines, automated deployments, environment promotion strategies, testing frameworks, and DevSecOps practices.

Certifications

Mandatory

• Databricks Certified Developer for Apache Spark

Preferred

• Databricks Data Analyst Associate
• AWS Solutions Architect Professional

Education

Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, Data Science, or related field.

Experience

10+ Years Overall IT Experience | 5+ Years Databricks Experience

Preferred Domains

Banking, Financial Services, SaaS Platforms, Data & AI Modernization Programs.

Required Skills

Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Lakeflow, DLT, Databricks Workflows, Databricks Asset Bundles (DAB), MLflow, Genie, Vector Search.

Expert knowledge of PySpark, Spark SQL, Python, SQL, Data Modeling, Data Warehousing, Kafka, and CDC architectures.