Azure Data Engineer

Antal International; Gurugram District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Antal International Gurugram seeks a visionary Lead Engineer – Azure Databricks to shape enterprise analytics. You will own cloud data architecture strategy, modernize analytics, and migrate legacy EDWs to a Databricks Lakehouse.

You will partner with IT leadership, business and analytics teams to deliver governed data solutions, enable AI/ML workloads, and drive cost-effective, high-performance data pipelines across the organization.

Qualifications

• 10+ years designing and delivering enterprise data & analytics platforms.
• Hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures.
• Advanced proficiency in SQL, Python, PySpark, Spark SQL.
• Strong experience modernizing legacy EDW environments.
• Knowledge of data governance, security, compliance, and quality.

Responsibilities

• Design and govern enterprise-scale cloud data platforms (data lakes, Lakehouse, EDW).
• Architect and implement Databricks Lakehouse solutions using Delta Lake and patterns.
• Lead migration of legacy EDW to Azure with phased strategies and integration.
• Establish architecture standards, best practices, CI/CD pipelines, and governance frameworks.
• Provide day-to-day technical leadership, mentoring, and prioritization.
• Deliver high-performance batch and real-time ingestion pipelines using PySpark and Databricks.

Skills

Azure Databricks

Enterprise data architecture

Leadership

Data governance

CI/CD

Mentoring

PySpark

SQL

Python

Education

Bachelor’s or Master in CS/Engineering/Data Science

Tools

Databricks

Delta Lake

Delta Live Tables

Power BI/Tableau (asset)

Job description

Are you a visionary cloud data architect ready to shape the future of enterprise analytics? We’re seeking a passionate Lead Engineer – Azure Databricks to join our high-impact Business Analytics/BI team as a senior technical leader. This is a career-defining opportunity to architect scalable, secure, and high-performance cloud data platforms that power reporting, advanced analytics, and AI/ML innovation across the organization.

In this strategic role, you will own the end-to-end cloud data architecture strategy, drive our analytics modernization journey, and lead the migration of legacy Enterprise Data Warehouses to a modern Azure Databricks Lakehouse ecosystem. You’ll partner closely with IT leadership, business stakeholders, and analytics teams to deliver best-in-class, governed data solutions that accelerate insights and business value.

Exp: 5 to 12 years

Location: Gurugram

Shift: 2-11pm

Key Responsibilities

• Design and govern enterprise-scale cloud data platforms (data lakes, Lakehouse, and cloud data warehouses) supporting BI/EDW, analytics, and AI/ML workloads
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Medallion (Bronze/Silver/Gold) patterns
• Lead the full lifecycle of legacy EDW migration to Azure, including phased strategies, data refactoring, and seamless integration with enterprise systems
• Establish architecture standards, best practices, CI/CD pipelines, and data governance frameworks (Unity Catalog, RBAC, encryption)
• Provide day-to-day technical leadership, mentoring, and prioritization to a team of data engineers and vendor partners
• Deliver high-performance batch and real-time ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables, and Structured Streamng
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products for BI teams

What You’ll Bring

• 10+ years designing and delivering enterprise data & analytics platforms, with proven hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data serices
• Strong experience modernizing legacy EDW environments and building governed, scalable data platforms
• Demonstrated leadership in architecture reviews, design discussions, and guiding cross-functional teams
• Solid understanding of data governance, security, compliance, and quality frameworks
• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or related field
• Experience with SAP ERP analytics integration (S/4HANA or ECC) is a strong asset; familiarity with Power BI/Tableau and Microsoft Fabric is a plus
• Bonus: Exposure to AI/ML-ready data platforms or Agentic AI concepts