AVP - Databricks Architect

Impetus Career Consultants; Hyderabad; On-site; INR 2,500,000 - 3,500,000; Full time

Job summary

A leading consulting firm in Hyderabad seeks a Talent Acquisition Specialist with over 15 years in data engineering. The role requires expertise in building scalable data pipelines and implementing data governance. Ideal candidates possess experience with Databricks, Azure Data Lake, and a strong background in ETL/ELT processes. This full-time position is based in the office with UK shift timings.

Qualifications

• 15+ years experience in Data Engineering.
• Deep hands-on experience with databricks and cloud data architecture.
• Experience in enterprise data governance, metadata, and lineage.

Responsibilities

• Architect, build, and optimize data pipelines using Databricks.
• Implement governance across data assets and operationalize ML models.
• Lead cross-functional data initiatives and enforce coding standards.

Skills

Data Engineering

Databricks Lakehouse

Azure Data Lake

ETL/ELT Pipeline Design

DevOps

Education

Azure Data Engineer / Cloud certifications

Tools

PySpark

Databricks

ADF

MLflow

Job description

Talent acquisition specialist | Ex- Tech M | Unleashing Talent and Transforming Futures! 🚀 #HRwithAI #RecruitmentRising

Shift Type: UK Shift

Work Model: 5 Days Work From Office

Lead the design and delivery of large-scale data engineering solutions on Azure and Databricks Lakehouse. This role focuses on scalable data pipelines, Unity Catalog governance, ML/LLM enablement, and driving engineering excellence.

Key Responsibilities

• Architect, build, and optimize data pipelines using Databricks, ADF, PySpark, and Unity Catalog.
• Implement data quality, lineage, privacy, and security governance across data assets.
• Integrate data from multiple sources to support analytics, ML feature stores, and enterprise insights.
• Operationalize ML models using MLflow and enhance productivity using LLM/Copilot solutions.
• Lead cross‑functional data initiatives end‑to‑end; mentor engineers and enforce coding & platform standards.
• Drive efficiency through CI/CD, DevOps, automation, and reusable framework components.

Required Qualifications

• 15+ years in Data Engineering with deep hands‑on experience in:
• Databricks Lakehouse, ADF, PySpark, Azure Data Lake, Unity Catalog
• Strong background in ETL/ELT pipeline design, orchestration, and cloud data architecture.
• Experience with enterprise data governance, metadata, and lineage capabilities.
• Azure Data Engineer / Cloud certifications preferred.

Seniority Level

Director

Employment Type

Full‑time

Job Function

Information Technology

Industries

IT Services and IT Consulting, Business Consulting and Services