Azure Data Engineer

Antal International; Gurugram District; On-site; INR 4,000,000 - 6,400,000; Full time

Job summary

Antal International in Gurugram seeks a Lead Engineer - Azure Databricks to architect scalable cloud data platforms for reporting, analytics and AI/ML. You will own end-to-end data architecture strategy and lead migration of legacy EDW to a modern Lakehouse using Databricks.

You will mentor a team of data engineers, collaborate with IT leadership and stakeholders, and drive governance, security, and performance optimization across the analytics stack.

Qualifications

• 9+ years designing and delivering enterprise data & analytics platforms.
• Hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures.
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data services.
• Strong experience modernizing legacy EDW environments with governed data platforms.
• Demonstrated leadership in architecture reviews and guiding cross-functional teams.
• Experience with data governance, security, compliance, and quality frameworks.

Responsibilities

• Design and govern enterprise-scale cloud data platforms supporting BI/EDW and AI/ML workloads.
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Bronze/Silver/Gold patterns.
• Lead migration of legacy EDW to Azure with phased strategies and integration with enterprise systems.
• Establish architecture standards, CI/CD pipelines, and data governance frameworks (Unity Catalog, RBAC, encryption).
• Provide day-to-day technical leadership, mentoring, and prioritization to a team of data engineers and vendor partners.
• Deliver high-performance batch and real-time ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables.
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products for BI teams.

Skills

Azure Databricks

Delta Lake

Lakehouse architectures

SQL

Python

PySpark

Data Governance

Leadership

Cloud security

Education

Bachelor's in Computer Science or Engineering

Master's in a related field

Tools

Azure

Power BI

Tableau

Microsoft Fabric

Unity Catalog

Job description

Are you a visionary cloud data architect ready to shape the future of enterprise analytics? We're seeking a passionate Lead Engineer - Azure Databricks to join our high-impact Business Analytics/BI team as a senior technical leader. This is a career-defining opportunity to architect scalable, secure, and high-performance cloud data platforms that power reporting, advanced analytics, and AI/ML innovation across the organization.

In this strategic role, you will own the end-to-end cloud data architecture strategy, drive our analytics modernization journey, and lead the migration of legacy Enterprise Data Warehouses to a modern Azure Databricks Lakehouse ecosystem. You'll partner closely with IT leadership, business stakeholders, and analytics teams to deliver best-in-class, governed data solutions that accelerate insights and business value.

Exp: 9 to 14 years

Location: Gurugram

Shift: 2-11pm

Key Responsibilities

• Design and govern enterprise-scale cloud data platforms (data lakes, Lakehouse, and cloud data warehouses) supporting BI/EDW, analytics, and AI/ML workloads
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Medallion (Bronze/Silver/Gold) patterns
• Lead the full lifecycle of legacy EDW migration to Azure, including phased strategies, data refactoring, and seamless integration with enterprise systems
• Establish architecture standards, best practices, CI/CD pipelines, and data governance frameworks (Unity Catalog, RBAC, encryption)
• Provide day-to-day technical leadership, mentoring, and prioritization to a team of data engineers and vendor partners
• Deliver high-performance batch and real-time ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables, and Structured Streamng
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products for BI teams

What You'll Bring

• 9+ years designing and delivering enterprise data & analytics platforms, with proven hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data serices
• Strong experience modernizing legacy EDW environments and building governed, scalable data platforms
• Demonstrated leadership in architecture reviews, design discussions, and guiding cross-functional teams
• Solid understanding of data governance, security, compliance, and quality frameworks
• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field
• Experience with SAP ERP analytics integration (S/4HANA or ECC) is a strong asset; familiarity with Power BI/Tableau and Microsoft Fabric is a plus
• Bonus: Exposure to AI/ML-ready data platforms or Agentic AI concepts