Databricks Arhitect

Saaki Argus & Averil Consulting; Chennai District; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

Saaki Argus & Averil Consulting seeks an Architect in Chennai to design and deliver enterprise-grade Lakehouse solutions using Databricks. You will build scalable batch and real-time pipelines with PySpark/SQL, implement Delta Lake features, and ensure metadata-driven ingestion.

You will lead data governance, security, and quality, work on ML lifecycle with MLflow, and drive CI/CD and DevOps for Spark workloads on Azure. Strong mentoring and stakeholder collaboration are expected.

Qualifications

• Architect and implement enterprise-grade Lakehouse solutions using Databricks.
• Design scalable batch and real-time data pipelines with Spark (PySpark/SQL).
• Build ETL/ELT pipelines, incremental loads, and metadata-driven ingestion.

Responsibilities

• Implement and optimize Databricks components: Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, and Workflows.
• Design large-scale data warehousing with 3NF and dimensional modeling.
• Establish data governance, security, and data quality with Unity Catalog.

Skills

Databricks

Apache Spark

PySpark

SQL

ETL/ELT

Delta Lake

Delta Live Tables

Autoloader

Structured Streaming

MLflow

Unity Catalog

CI/CD

DevOps

Azure

Data Governance

3NF modeling

Dimensional modeling

ServiceNow integration

Workflows

Tools

Azure Data Factory

Job description

Job Title: Architect

Location: Chennai

Key Responsibilities

• Architect and implement enterprise-grade Lakehouse solutions using Databricks
• Design and deliver scalable batch and real-time data pipelines using Apache Spark (PySpark/SQL)
• Build ETL/ELT pipelines, incremental data loads, and metadata-driven ingestion frameworks
• Implement and optimize Databricks components: Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, and Workflows
• Design large-scale data warehousing solutions with 3NF and dimensional modeling
• Establish data governance, security, and data quality frameworks, including Unity Catalog
• Lead ML lifecycle management using MLflow and drive AI use cases (RAG, AI/BI)
• Manage cloud-native deployments on Microsoft Azure and integrate with enterprise systems (e.g., ServiceNow)
• Drive CI/CD, DevOps practices, and performance optimization of Spark workloads
• Provide technical leadership, mentor teams, and ensure successful delivery
• Collaborate with stakeholders to translate business requirements into scalable solutions