Azure Data Engineer

Antal International; Gurugram District; Vor Ort; INR 4.500.000 - 7.000.000; Vollzeit

Zusammenfassung

Antal International is seeking a visionary Lead Engineer - Azure Databricks to architect scalable, secure cloud data platforms powering reporting, analytics, and AI/ML across the organization.

You will own the end-to-end cloud data architecture strategy, drive analytics modernization, and lead migration of legacy EDWs to a modern Lakehouse ecosystem on Azure, partnering with IT leadership and analytics teams to deliver governance and value.

Qualifikationen

• 9+ years designing and delivering enterprise data & analytics platforms with hands-on Azure Databricks, Delta Lake, and Lakehouse architectures.
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data services.
• Experience modernizing legacy EDW environments with governed, scalable data platforms.

Aufgaben

• Design and govern enterprise-scale cloud data platforms and orchestrate BI/EDW, analytics, and AI/ML workloads.
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Bronze/Silver/Gold patterns.
• Lead migration of legacy EDW to Azure with phased strategies and dataRefactoring.
• Establish architecture standards, CI/CD pipelines, and data governance (Unity Catalog, RBAC, encryption).
• Provide day-to-day technical leadership to a team of data engineers and vendor partners.
• Deliver high-performance ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables, and Structured Streaming.
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products.

Kenntnisse

Azure Databricks

Delta Lake

Lakehouse architectures

SQL

Python

PySpark

Spark SQL

Leadership

Data governance

EDW modernization

Ausbildung

Bachelor's or Master's degree in Computer Science, Engineering, Data Science

Tools

Databricks

Power BI

Tableau

Jobbeschreibung

Are you a visionary cloud data architect ready to shape the future of enterprise analytics? We're seeking a passionate Lead Engineer - Azure Databricks to join our high-impact Business Analytics/BI team as a senior technical leader. This is a career-defining opportunity to architect scalable, secure, and high-performance cloud data platforms that power reporting, advanced analytics, and AI/ML innovation across the organization.

In this strategic role, you will own the end-to-end cloud data architecture strategy, drive our analytics modernization journey, and lead the migration of legacy Enterprise Data Warehouses to a modern Azure Databricks Lakehouse ecosystem. You'll partner closely with IT leadership, business stakeholders, and analytics teams to deliver best-in-class, governed data solutions that accelerate insights and business value.

Exp: 9 to 14 years

Location: Gurugram

Shift: 2-11pm

Key Responsibilities

• Design and govern enterprise-scale cloud data platforms (data lakes, Lakehouse, and cloud data warehouses) supporting BI/EDW, analytics, and AI/ML workloads
• Architect and implement Databricks Lakehouse solutions using Delta Lake and Medallion (Bronze/Silver/Gold) patterns
• Lead the full lifecycle of legacy EDW migration to Azure, including phased strategies, data refactoring, and seamless integration with enterprise systems
• Establish architecture standards, best practices, CI/CD pipelines, and data governance frameworks (Unity Catalog, RBAC, encryption)
• Provide day-to-day technical leadership, mentoring, and prioritization to a team of data engineers and vendor partners
• Deliver high-performance batch and real-time ingestion pipelines using PySpark, SQL, Databricks Workflows, Delta Live Tables, and Structured Streamng
• Optimize cost, performance, and reliability while enabling self-service analytics and trusted data products for BI teams

What You'll Bring

• 9+ years designing and delivering enterprise data & analytics platforms, with proven hands-on expertise in Azure Databricks, Delta Lake, and Lakehouse architectures
• Advanced proficiency in SQL, Python, PySpark, Spark SQL, and modern cloud data serices
• Strong experience modernizing legacy EDW environments and building governed, scalable data platforms
• Demonstrated leadership in architecture reviews, design discussions, and guiding cross-functional teams
• Solid understanding of data governance, security, compliance, and quality frameworks
• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field
• Experience with SAP ERP analytics integration (S/4HANA or ECC) is a strong asset; familiarity with Power BI/Tableau and Microsoft Fabric is a plus
• Bonus: Exposure to AI/ML-ready data platforms or Agentic AI concepts