Databricks - Solution Architect

Hoonartek; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Hoonartek is hiring a Databricks Architect to own the organization’s data architecture, oversee implementation quality, and ensure scalable, secure, and cost-effective Databricks solutions. You will lead architecture reviews, governance, and performance optimizations across Delta Lake, Unity Catalog, and Delta Live Tables.

The role requires deep expertise across the Lakehouse platform, cloud provider experience, and strong collaboration with delivery partners to drive successful outcomes.

Qualifications

• 10–15 years IT experience with Databricks Lakehouse architecture.
• Hands-on with Delta Lake, Unity Catalog, and Delta Live Tables.
• Experience with at least one cloud provider (Azure/AWS/GCP).
• Strong performance tuning for Spark workloads.
• CICD and IaC practices for Databricks.
• Excellent communication with cross-functional teams.

Responsibilities

• Own and maintain the current and target-state Databricks Lakehouse architecture: Delta Lake structures, Unity Catalog metastore/catalog design, Workflows/job orchestration, Delta Live Tables pipelines, and cluster/workspace topology.
• Define and govern Unity Catalog structure — catalogs, schemas, access-control models (RBAC/ABAC, row/column-level security), data lineage, and audit logging.
• Review pipeline and notebook architecture (DLT, Workflows/Jobs, orchestration patterns) for scalability, reliability, and maintainability.
• Set and enforce cluster policies, sizing, and autoscaling standards; monitor DBU consumption, Photon/serverless usage, and job efficiency to control platform cost.
• Define the platform's security architecture: encryption, network isolation (VNet/VPC peering, private link), IAM integration, and compliance alignment.
• Establish CI/CD and IaC standards for Databricks — Asset Bundles, Repos/Git integration.
• Design and evolve the data governance and metadata strategy (lineage, cataloging, quality checks) across the Lakehouse.
• Guide integration patterns with upstream/downstream systems — source ingestion, and consumption layers for BI, ML, and APIs.
• Lead performance tuning and troubleshooting for Spark jobs, query optimization, and workload management.
• Maintain architecture documentation, design decision records, and the platform roadmap.
• Review and sign off on deliverables from the implementation vendor to confirm they meet these architecture, security, and performance standards.

Skills

Databricks Lakehouse

Unity Catalog governance

Delta Live Tables

Spark performance tuning

DBSQL/Photon

Cloud platforms (Azure/AWS/GCP)

ETL/ELT data engineering

CI/CD with Asset Bundles

Job description

We empower enterprises globally through intelligent, creative, and insightful services for data integration, data analytics and data visualization.

Hoonartek is a leader in enterprise transformation, data engineering and an acknowledged world-class Ab Initio delivery partner.

Using centuries of cumulative experience, research and leadership, we help our clients eliminate the complexities & risk of legacy modernization and safely deliver big data hubs, operational data integration, business intelligence, risk & compliance solutions and traditional data warehouses & marts.

At Hoonartek, we work to ensure that our customers, partners and employees all benefit from our unstinting commitment to delivery, quality and value. Hoonartek is increasingly the choice for customers seeking a trusted partner of vision, value and integrity

How We Work?

Define, Design and Deliver (D3) is our in-house delivery philosophy. It’s culled from agile and rapid methodologies and focused on ‘just enough design’. We embrace this philosophy in everything we do, leading to numerous client success stories and indeed to our own success.

We embrace change, empowering and trusting our people and building long and valuable relationships with our employees, our customers and our partners. We work flexibly, even adopting traditional/waterfall methods where circumstances demand it. At Hoonartek, the focus is always on delivery and value.

Job Description

Role

We are hiring a Databricks Architect with deep expertise across the Databricks Lakehouse Platform to own the organization's data architecture — current state and target state — and provide technical oversight of the implementation delivered, signing off on its design and quality.

Key Responsibilities

• Own and maintain the current and target-state Databricks Lakehouse architecture: Delta Lake structures, Unity Catalog metastore/catalog design, Workflows/job orchestration, Delta Live Tables pipelines, and cluster/workspace topology.
• Define and govern Unity Catalog structure — catalogs, schemas, access-control models (RBAC/ABAC, row/column-level security), data lineage, and audit logging.
• Review pipeline and notebook architecture (DLT, Workflows/Jobs, orchestration patterns) for scalability, reliability, and maintainability.
• Set and enforce cluster policies, sizing, and autoscaling standards; monitor DBU consumption, Photon/serverless usage, and job efficiency to control platform cost.
• Define the platform's security architecture: encryption, network isolation (VNet/VPC peering, private link), IAM integration, and compliance alignment.
• Establish CI/CD and IaC standards for Databricks — Asset Bundles, Repos/Git integration.
• Design and evolve the data governance and metadata strategy (lineage, cataloging, quality checks) across the Lakehouse.
• Guide integration patterns with upstream/downstream systems — source ingestion, and consumption layers for BI, ML, and APIs.
• Lead performance tuning and troubleshooting for Spark jobs, query optimization, and workload management.
• Maintain architecture documentation, design decision records, and the platform roadmap.
• Review and sign off on deliverables from the implementation vendor to confirm they meet these architecture, security, and performance standards.

Required Skills and Experience

• 10-15 years of overall IT experience, with deep hands-on and architecture-level expertise across the Databricks Lakehouse Platform: Delta Lake, Unity Catalog, Delta Live Tables, Workflows, DBSQL, Photon, and MLflow.
• Strong command of at least one hyperscaler running Databricks (Azure, AWS, or GCP).
• Hands-on depth in Unity Catalog governance: catalog/schema/table design, access-control models, data lineage, and audit logging.
• Strong grasp of Spark internals, performance tuning, and cluster/job optimization.
• Working knowledge of Databricks CI/CD and IaC tooling — Asset Bundles, Repos/Git integration.
• Solid data engineering fundamentals: ETL/ELT pipeline design, data modeling, and orchestration.
• Experience with DBU-based cost governance/FinOps for Databricks.
• Excellent communication skills to work with cross-functional stakeholders and external delivery partners.
• Preferred: Databricks Certified Data Engineer Professional or Databricks Certified Solutions Architect; a cloud architecture certification (Azure/AWS).