Databricks Data Architect Databricks Data Architect Experience- 10 to 12 Years

Hoonartek; Mumbai; On-site; INR 2,000,000 - 3,000,000; Full time

Job summary

Hoonartek is seeking an experienced Senior Data Architect in Mumbai, India, to design and implement scalable and secure data platforms. This role involves defining architecture roadmaps, guiding the implementation of data solutions using Databricks, and ensuring alignment with business objectives.

The ideal candidate will have a strong background in data engineering and architecture, particularly in leveraging cloud platforms like Azure, and possess expertise in data governance and security.

Qualifications

• 10+ years of experience in data engineering/architecture.
• 4+ years of hands-on Databricks experience.
• Strong expertise in PySpark, SQL, Python, Delta Lake.

Responsibilities

• Define enterprise data architecture roadmap.
• Design modern data platforms using Databricks.
• Implement data governance frameworks.

Skills

Databricks

PySpark

SQL

Delta Lake

Cloud Platforms (Azure/AWS/GCP)

Data Architecture

Data Governance

AI/ML/LLM integrations

Education

Bachelor’s/Master’s degree in Computer Science or Engineering

Tools

Databricks Apps

Airflow

Azure Data Factory

Job description

Job Description

We are looking for an experienced Senior Data Architect to design and lead the implementation of scalable, secure, and high-performance data platforms. The ideal candidate will have strong experience in Databricks, modern data architectures (Lakehouse), cloud platforms (Azure/AWS/GCP), and enterprise data strategy.

This role will drive data architecture decisions, enable advanced analytics and AI use cases, and ensure alignment with business objectives.

Key Responsibilities

1. Data Architecture & Strategy

• Define and implement enterprise data architecture roadmap
• Design modern data platforms (Lakehouse architecture) using Databricks
• Lead architecture for data ingestion, transformation, storage, and consumption layers
• Design Databricks workspace architecture, including cluster sizing and workload-based compute strategy and workspace-level governance (Unity Catalog, access isolation)
• Architect job orchestration and scheduling frameworks (Workflows / external schedulers)
• Enable integration with Azure Key Vault/Secrets management systems, external APIs and enterprise systems, AI/LLM services for advanced analytics use cases
• Perform capacity planning, sizing, and cost estimation for Databricks workloads
• Architect and optimize solutions using Databricks (Delta Lake, Unity Catalog, Workflows)
• Design ETL/ELT pipelines using PySpark, SQL, and notebooks
• Implement Delta Lake features (ACID transactions, time travel, schema evolution)
• Drive adoption of Databricks best practices and performance optimization
• Design and validate cloud landing zone architecture for data platform
• Drive the infrastructure sizing and cost optimization across compute, storage, and networking
• Ensure secure data access patterns and network isolation for enterprise data platforms
• Design scalable solutions in Azure (preferred), AWS, or GCP
• Integrate Databricks with services such as: Azure Data Factory / AWS Glue
  ADLS / S3 / BigQuery
• Build real-time and batch data processing pipelines

4. Data Governance & Security

• Implement data governance frameworks using Unity Catalog or similar tools
• Ensure data quality, lineage, cataloging, and compliance
• Define and enforce data security and access management

5. Stakeholder Management

• Collaborate with business stakeholders, data scientists, and engineering teams
• Translate business requirements into scalable technical solutions
• Provide architectural leadership across multiple projects

6. Performance & Optimization

• Optimize data pipelines, storage, and compute costs
• Implement monitoring, logging, and performance tuning mechanismsEnsure high availability and scalability of data systems
• Mentor and guide data engineers and architects
• Establish coding standards, best practices, and reusable frameworks
• Lead design reviews and architecture governance forums

Required Qualifications

• Bachelor’s/Master’s degree in Computer Science, Engineering, or related field
• 10+ years of experience in data engineering / architecture
• 4+ years of hands‑on Databricks experience
• Strong expertise in PySpark, SQL, Python, Delta Lake and Lakehouse architecture, Distributed data processing

Must‑Have Technical Skills

• Hands‑on Knowledge of Databricks – Delta Lake, Unity Catalog, Lakeflow, Lakebase, Databricks Apps, Workflows, job scheduling, ETL/ELT pipeline orchestration
• Strong understanding of Databricks architecture (control plane vs data plane, workspace model)
• Experience with cluster sizing, autoscaling, workload-based optimisation, cost estimation and performance tuning, FinOps
• Building CI/CD pipelines and Declarative Automation Bundle (DAB) implementation
• Exposure to AI/ML/LLM integrations within Databricks ecosystem (MLflow, external LLM APIs)
• Strong understanding of cloud landing zone concepts (subscription design, governance, policies)
• Understanding of enterprise security architecture for data platforms
• Data modelling: Dimensional, Data Vault
• Data warehousing
• API and data integration patterns

Good‑to‑Have Technical Skills

• Understanding of cloud network topology for secure data platforms – VNet injection, private endpoints, subnet isolation, NSG, secure access to storage, Databricks, and external systems
• Knowledge of hybrid connectivity models – On‑premises integration (VPN, ExpressRoute), cross‑cloud architecture (AWS/Azure/GCP interoperability)
• Identity and access management, secret management
• Knowledge and hands‑on with data governance tools (Collibra, Alation or equivalent)
• Real‑time streaming design and implementation