Senior Data Engineer

Antal International; Gurugram District; On-site; INR 2,500,000 - 3,500,000; Full time

Job summary

A leading analytics firm is seeking a passionate Lead Engineer – Azure Databricks to shape enterprise analytics. This role involves architecting scalable, secure, and high-performance cloud data platforms for advanced analytics. With 10+ years of experience in enterprise data and proven expertise in Azure Databricks, you will lead the migration to a modern Lakehouse ecosystem. The ideal candidate has strong SQL and Python skills, experience in data governance, and a relevant degree. Join us to drive analytics modernization and deliver high-performance solutions.

Qualifications

• 10+ years designing and delivering enterprise data & analytics platforms.
• Proven expertise in Azure Databricks and Delta Lake architectures.
• Strong experience in modernizing legacy EDW environments.

Responsibilities

• Design and govern enterprise-scale cloud data platforms.
• Lead the migration of legacy Enterprise Data Warehouses to Azure.
• Provide technical leadership and mentoring to a team of data engineers.

Skills

Azure Databricks

SQL

Python

PySpark

Data governance

AI/ML platforms

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Databricks

Delta Lake

SAP ERP

Power BI

Tableau

Job description

Are you a visionary cloud data architect ready to shape the future of enterprise analytics? We’re seeking a passionate Lead Engineer – Azure Databricks to join our high-impact Business Analytics/BI team as a senior technical leader. This is a career-defining opportunity to architect scalable, secure, and high-performance cloud data platforms that power reporting, advanced analytics, and AI/ML innovation across the organization.

In this strategic role, you will own the end-to-end cloud data architecture strategy, drive our analytics modernization journey, and lead the migration of legacy Enterprise Data Warehouses to a modern Azure Databricks Lakehouse ecosystem. You’ll partner closely with IT leadership, business stakeholders, and analytics teams to deliver best-in-class, governed data solutions that accelerate insights and business value.

Exp: 9 to 13 years

Location: Gurugram

Shift: 2-11pm

Key Responsibilities

• Design and govern enterprise-scale cloud data platforms (data lakes, Lakehouse, and cloud data warehouses) supporting BI/EDW, analytics, and AI/ML workloads
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Medallion (Bronze/Silver/Gold) patterns
• Lead the full lifecycle of legacy EDW migration to Azure, including phased strategies, data refactoring, and seamless integration with enterprise systems
• Establish architecture standards, best practices, CI/CD pipelines, and data governance frameworks (Unity Catalog, RBAC, encryption)
• Provide day-to-day technical leadership, mentoring, and prioritization to a team of data engineers and vendor partners
• Deliver high-performance batch and real-time ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables, and Structured Streaming
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products for BI teams

What You’ll Bring

• 10+ years designing and delivering enterprise data & analytics platforms, with proven hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data services
• Strong experience modernizing legacy EDW environments and building governed, scalable data platforms
• Demonstrated leadership in architecture reviews, design discussions, and guiding cross-functional teams
• Solid understanding of data governance, security, compliance, and quality frameworks
• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or related field
• Experience with SAP ERP analytics integration (S/4HANA or ECC) is a strong asset; familiarity with Power BI/Tableau and Microsoft Fabric is a plus
• Bonus: Exposure to AI/ML-ready data platforms or Agentic AI concepts