Databricks PlatformLead

Domnic Lewis; Navi Mumbai; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Unknown company is seeking a seasoned data platform leader to own the enterprise data platform strategy and roadmap. You will guide transitions to modern lakehouse architectures and lead cross‑functional delivery across engineering, analytics, IT, and business teams.

You will design and implement data pipelines, governance, and observability, while driving cost efficiency and reliability in Fabric, Databricks and Snowflake ecosystems.

Qualifications

• Bachelor’s degree in CS, SE, IS or related field.
• 10+ years in data engineering or architecture.
• 3–5 years in senior technical leadership.
• Hands-on with Fabric, Databricks or Snowflake in enterprise contexts.

Responsibilities

• Own the platform strategy and roadmap for enterprise data platforms.
• Lead end-to-end data platform projects with cross-functional teams.
• Define architecture, standards and governance for data pipelines.
• Oversee migration and decommissioning of legacy systems and on-prem data sources.

Skills

Data engineering

Data platforms

Enterprise architecture

Azure Databricks

Snowflake

Microsoft Fabric

SQL & ETL/ELT

Education

Bachelor's in CS/IS/SE

Master's in CS/Data Science

Tools

Microsoft Fabric

Databricks

Snowflake

Azure Data Factory

Palantir Foundry

Delta Live Tables

dbt

Azure DevOps

Job description

Function: Information Management / IT

Key Accountabilities

1. Platform Strategy & Technical Leadership

• Own the enterprise data platform strategy and roadmap, guiding the transition from legacy platforms to Microsoft Fabric, Databricks and/or Snowflake.
• Act as the primary technical decision-maker for data platform tools, standards and architecture choices.
• Lead architecture and design reviews, establishing engineering quality standards.
• Represent data platform engineering in leadership forums and vendor discussions.
• Stay current with data integration, automation and emerging data platform technologies and apply them where they deliver business value.

2. Project Leadership & Cross-Functional Delivery

• Lead major data platform projects end-to-end, coordinating across engineering, analytics, IT and business teams.
• Serve as the technical lead for enterprise data migration programs.
• Follow a value-first delivery approach by prototyping early, reducing data debt and ensuring operational readiness.
• Create and maintain architecture documents, runbooks, technical documentation and project status updates.
• Design and implement enterprise Lakehouse, Data Lake and Data Warehouse architectures.
• Build and maintain enterprise Medallion Architecture (Bronze–Silver–Gold).
• Lead migration and decommissioning of SAP BW and on-premise SQL platforms, including historical data loads, validation and cutover.
• Define and enforce platform standards, reusable templates and engineering guardrails.
• Design integration patterns for Palantir Foundry and Aera within the Fabric/Databricks ecosystem.
• Build enterprise-grade data pipelines using Microsoft Fabric Pipelines, Azure Data Factory and Databricks.
• Integrate SAP, CRM, ERP, IoT and SQL data sources using appropriate extraction and integration patterns.
• Implement real-time data pipelines using Eventstream and/or Event Hubs.
• Own transformation logic using Delta Live Tables, dbt and Fabric Notebooks.
• Reduce data duplication through virtualization and data-sharing patterns such as Shortcuts and Delta Sharing.
• Own pipeline monitoring, SLAs, alerting and incident response.
• Define CI/CD standards using Azure DevOps and Databricks Asset Bundles.
• Implement automated data testing frameworks using DLT checks, Great Expectations and dbt tests.
• Continuously improve platform performance, cost efficiency, reliability and observability.
• Establish documentation standards and leverage AI-powered tools to streamline engineering documentation.

6. Data Products, Governance & Observability

• Design and deliver governed data products with clear ownership, SLAs and data contracts.
• Implement automated data quality controls across ingestion and transformation layers.
• Build and maintain enterprise data catalog and lineage using Microsoft Purview and Unity Catalog.
• Deploy data observability capabilities covering schema drift, data freshness, volume anomalies, lineage and cost monitoring.
• Define systems of record and enforce appropriate RBAC, RLS and data masking controls.
• Provide high-quality, governed and AI-ready data for ML, RAG and agentic AI workloads.
• Build data preparation pipelines supporting preprocessing, embeddings and retrieval use cases.
• Integrate Palantir Foundry and its ontology with Fabric/Databricks.
• Champion AI-powered engineering tools such as GitHub Copilot, Databricks AI Assistant and Azure AI.

Candidate Profile

Education

Required:

• Bachelor's degree in Computer Science, Software Engineering, Information Systems, Data Engineering or a closely related technical discipline.

Preferred:

• Master's degree in Computer Science, Data Science or Information Management.
• Equivalent professional experience and relevant certifications may be considered in lieu of an advanced degree.

Experience

Required:

• 10+ years of hands-on experience in data engineering, data platforms or data architecture.
• 3–5 years at a senior/principal technical level, owning platform strategy and enterprise-scale technical direction.
• Proven experience leading complex, multi-workstream data platform projects end-to-end across engineering, analytics, IT and business teams.
• Experience owning and delivering an enterprise data platform strategy, including architecture decisions, technology selection and engineering standards.
• Hands-on experience with Microsoft Fabric, Databricks and/or Snowflake in enterprise Lakehouse environments.
• Strong experience designing enterprise data architectures covering Lakehouse, Data Lake, Data Warehouse and Medallion architectures.
• Proven experience executing on-premise-to-cloud migrations, including source mapping, historical data loads, validation and cutover.
• Experience establishing data governance foundations covering catalog, lineage, data quality, access controls and ownership models.
• Strong track record delivering multiple data integration patterns including ETL/ELT, replication, virtualization and streaming/event-based pipelines.
• Demonstrated experience with pipeline monitoring, data observability and incident management, including SLA ownership and root-cause analysis.
• Experience defining engineering standards and reusable frameworks for broader data engineering organizations.

Preferred:

• Experience working in manufacturing, chemicals or process-industry environments.
• Experience designing and delivering self-service or federated data products.
• Exposure to Palantir Foundry, including ontology, pipelines and AIP integration.
• Experience enabling data science/ML teams through feature pipelines, MLOps and model-scoring pipelines.

Preferred Certifications

• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)