Databricks Architect

Infovision Labs; Pune District, Bengaluru; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Infovision Labs seeks an experienced Enterprise Data Architect with 10+ years in data architecture and 5+ years on Databricks to design and lead enterprise-scale data solutions. The role requires hands-on Databricks expertise (PySpark, Delta Live Tables), dbt, and cloud data services.

You will drive architecture decisions, mentor teams, and ensure governance and best practices across projects. The ideal candidate collaborates with onshore/offshore teams, defines ELT/ETL pipelines, and optimizes

Qualifications

• At least 10+ years of enterprise data architecture experience.
• 5+ years of relevant data engineering experience on Databricks.
• Strong expertise in Databricks components: Workspace, Clusters, Jobs, Repos, Delta Live Tables.

Responsibilities

• Lead the design and implementation of scalable, secure, high-performing data solutions using Azure Databricks, Delta Lake, and Delta Live Tables.
• Own the complete architecture process, including requirements gathering, solution design, documentation, technical reviews and implementing advanced data solutions on the Databricks platform.
• Define best practices for cloud data architecture, data modeling, ELT/ETL pipelines, workspace setup, cluster management, repos, and job orchestration.
• Coordinate and communicate with onshore and offshore teams, including end-users, data engineers, reporting specialists, and business analysts.
• Ensure solution compliance with data privacy, security, and governance standards.
• Conduct performance tuning and optimization of Databricks clusters.
• Ensure data quality, lineage, and observability across all pipelines.
• Monitor and troubleshoot data pipelines to ensure data quality and reliability.
• Lead the integration of Databricks with other data platforms and tools.
• Create processes and workflows to support data solutions documents, and lead solution reviews and audits for quality.
• Stay updated with industry best practices, Databricks and Azure developments, and introduce improvements to the technical stack.

Skills

Databricks

PySpark

dbt

Delta Lake

Delta Live Tables

Azure Databricks

Data modeling

CI/CD

DevOps

Communication

Tools

Git

SQL

Synapse

ADF

MLflow

Delta Live Tables

Job description

Job Details

We are seeking an experienced Architect with over 10+ years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead enterprise-scale data solutions. The ideal candidate is hands-on with Databricks ecosystem, PySpark (UDFs, Pandas UDFs), and dbt, and is capable of driving architecture decisions, working with engineering teams, and ensuring best practices in data engineering. The architect will collaborate closely with onshore and offshore teams to ensure both technical and business objectives are met.

Roles & Responsibilities

• Lead the design and implementation of scalable, secure, and high-performing data solutions using Azure Databricks, Delta Lake, and Delta Live Tables.
• Own the complete architecture process, including requirements gathering, solution design, documentation, technical reviews and implement advanced data solutions on the Databricks platform.
• Define best practices for cloud data architecture, data modeling, ELT/ETL pipelines, workspace setup, cluster management, repos, and job orchestration.
• Coordinate and communicate with onshore and offshore teams, including end-users, data engineers, reporting specialists, and business analysts.
• Ensure solution compliance with data privacy, security, and governance standards.
• Conduct performance tuning and optimization of Databricks clusters.
• Ensure data quality, lineage, and observability across all pipelines.
• Monitor and troubleshoot data pipelines to ensure data quality and reliability.
• Lead the integration of Databricks with other data platforms and tools.
• Create processes and workflows to support data solutions documents, and lead solution reviews and audits for quality
• Stay updated with industry best practices, Databricks and Azure developments, and introduce improvements to the technical stack.

Job Description

• At least 10+ years of experience in enterprise data architecture, including design and implementation of large-scale data platforms and 5+ years of relevant data engineering experience on Databricks.
• Strong expertise in: Databricks (Workspace, Clusters, Jobs, Repos, Delta Live Tables)
• Deep hands-on expertise with Azure Databricks, PySpark, Delta Lake, Unity catalog, MLflow, DBT, and associated Azure data services (Data Lake, SQL, Synapse, ADF).
• Proven experience migrating from legacy data warehouse and reporting systems to modern cloud platforms.
• Experience in data modeling, data warehousing (OLTP, OLAP), security, governance, DevOps, and MLOps.
• Hands-on with CI/CD, version control (Git), and DevOps practices for data engineering
• Excellent communication, collaboration, problem-solving and analytical skills with the ability to collaborate effectively with cross-functional teams and influence decision-making at all levels of the organization.
• Ability to mentor and coach junior architects, engineers, and BI/reporting developers.
• Architect-level certifications in Databricks and DBT are preferred.