Databricks architect

Atain; India; Presencial; INR 3 500 000 - 7 000 000; Tempo integral

Resumo da oferta

Atain is seeking a senior, hands-on Databricks Architect to lead the design and delivery of an enterprise data lakehouse. You will establish a secure, scalable Databricks platform and drive adoption across data engineering, governance, analytics, and AI readiness.

You will own the lakehouse architecture, design pipelines with Delta Lake, unify data governance with Unity Catalog, and optimize performance and cost while guiding client stakeholders through architecture decisions and production

Qualificações

• Experience designing lakehouse architecture and enterprise Databricks platforms.
• Strong hands-on expertise in Spark, PySpark, Python, SQL and Delta Lake.
• Experience with Unity Catalog governance and data security.
• Experience with AWS or Azure, including security, networking, storage.
• Proven ability to lead architecture decisions and production readiness, with clear trade-offs.

Responsabilidades

• Own lakehouse architecture roadmap, including workspaces, cloud integration, storage, networking, identity, and compute.
• Design batch, streaming, and CDC pipelines using Delta Lake, Auto Loader, and Lakeflow with layered data architecture.
• Establish Unity Catalog governance covering access controls, lineage, discovery, and auditability.
• Enable Databricks SQL, BI integration, governed metrics, and assess Genie/AI/ML capabilities against business needs.
• Lead performance and cost optimization, CI/CD, infrastructure automation, monitoring, failure recovery, and disaster recovery.
• Guide client stakeholders and engineering teams through architecture decisions, production readiness, and knowledge transfer.
• Demonstrate ownership of enterprise Databricks platforms delivered into production.
• Strong client-facing communication and technical leadership with documented outcomes.

Conhecimentos

Spark

PySpark

Python

SQL

Delta Lake

Unity Catalog

Pipeline orchestration

AWS

Azure

CI/CD

Infrastructure as Code

Ferramentas

Terraform

Descrição da oferta de emprego

We are looking for a senior, hands-on Databricks Architect to lead the design and delivery of an enterprise data lakehouse. The role will establish a secure, scalable platform and drive adoption of Databricks capabilities across data engineering, governance, analytics, and AI readiness.

Key Responsibilities

• Own lakehouse architecture and implementation roadmap, including workspaces, cloud integration, storage, networking, identity, and compute.
• Design batch, streaming, and CDC pipelines using Delta Lake, Auto Loader, and Lakeflow, with layered data architecture, modeling, data quality, and reliable incremental processing.
• Establish Unity Catalog governance covering access controls, sensitive-data protection, lineage, discovery, and auditability.
• Enable Databricks SQL, BI integration, curated data products, and governed business metrics; assess Genie and AI/ML capabilities against business needs.
• Lead performance and cost optimization, CI/CD, infrastructure automation, monitoring, failure recovery, and disaster recovery.
• Guide client stakeholders and engineering teams through architecture decisions, implementation reviews, production readiness, and knowledge transfer.
• Proven ownership of enterprise Databricks platforms delivered into production.
• Strong hands-on expertise in Spark, PySpark, Python, SQL, Delta Lake, Unity Catalog, and pipeline orchestration.
• Experience with lakehouse modeling, streaming, CDC, schema evolution, deduplication, and safe pipeline recovery.
• Strong AWS or Azure expertise, including platform security, networking, storage, and enterprise integration.
• Demonstrated performance troubleshooting, cost optimization, automated testing, CI/CD, and infrastructure-as-code experience.
• Strong client-facing communication and technical leadership, with clear examples of architecture trade-offs and production outcomes.

Preferred

• Databricks Data Engineer Professional certification.
• Experience with serverless compute, Terraform, Declarative Automation Bundles, and warehouse migration.
• Exposure to AI/BI Genie, MLflow, model serving, vector search, or RAG on governed enterprise data.

Ideal Candidate

A hands-on architect who can build the lakehouse foundation, lead delivery, and expand Databricks adoption in line with business priorities.