Data Engineering Lead

Verdantas; Pune District; On-site; INR 350,000 - 650,000; Full time

Job summary

Verdantas is seeking a Data Engineering Lead in Pune to design and deliver scalable data solutions that support AI/ML, analytics, and web apps. You will architect batch and real-time data systems on Azure and Microsoft Fabric, lead ETL/ELT pipelines, and mentor a team of data engineers.

The ideal candidate has 8+ years in data engineering with 2+ in leadership, strong SQL/Python, and hands-on Azure Fabric/Azure Data Factory experience.

Qualifications

• Bachelor's or master's degree in computer science or related field.
• 8+ years of Data Engineering experience with 2+ years in a leadership role.
• Strong SQL and Python skills.
• Experience with Spark, Flink, ETL/ELT frameworks, and modern data architectures.
• Hands-on experience with Microsoft Azure services such as Azure Data Factory, Synapse, Data Lake, and Databricks.
• Experience with Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Power BI integration).
• Experience building data solutions for reporting, analytics, dashboards, and web applications.

Responsibilities

• Design and implement batch and real-time data architectures using modern cloud platforms.
• Define data engineering standards, governance, and best practices.
• Evaluate and recommend technologies for data ingestion, processing, storage, and reporting.
• Lead development of ETL/ELT pipelines using Spark, Airflow, dbt, Azure Data Factory, and Microsoft Fabric.
• Build and manage data lakes, lakehouses, data warehouses, and data marts.
• Ensure data quality, lineage, observability, and governance.
• Develop scalable data solutions supporting enterprise reporting, dashboards, and web applications.
• Mentor data engineers and drive technical excellence.
• Collaborate with business stakeholders, analysts, data scientists, and application teams.
• Manage project delivery, timelines, and resource allocation.
• Build and manage cloud-based data platforms on Azure and Microsoft Fabric.
• Implement CI/CD pipelines and Infrastructure as Code (IaC).
• Optimize performance, scalability, and cost efficiency.

Skills

SQL

Python

Communication

Stakeholder management

Education

Bachelor's or master's degree in computer science or related field

Tools

Azure

Azure Data Factory

Azure Synapse

Azure Databricks

Microsoft Fabric

Data Lake

Power BI integration

Job description

We at Verdantas seeking for a Data Engineering Lead to design and deliver scalable, secure, and modern data solutions that support AI/ML, reporting, analytics, and web applications. The ideal candidate will have strong expertise in Microsoft Azure, Microsoft Fabric, data platform architecture, and team leadership.

Key Responsibilities

Architecture & Strategy

• Design and implement batch and real-time data architectures using modern cloud platforms.
• Define data engineering standards, governance, and best practices.
• Evaluate and recommend technologies for data ingestion, processing, storage, and reporting.

Development & Operations

• Lead development of ETL/ELT pipelines using Spark, Airflow, dbt, Azure Data Factory, and Microsoft Fabric.
• Build and manage data lakes, lakehouses, data warehouses, and data marts.
• Ensure data quality, lineage, observability, and governance.
• Develop scalable data solutions supporting enterprise reporting, dashboards, and web applications.
• Mentor data engineers and drive technical excellence.
• Collaborate with business stakeholders, analysts, data scientists, and application teams.
• Manage project delivery, timelines, and resource allocation.
• Build and manage cloud-based data platforms on Azure and Microsoft Fabric.
• Implement CI/CD pipelines and Infrastructure as Code (IaC).
• Optimize performance, scalability, and cost efficiency.

Skills & Qualifications

• Bachelor's or master's degree in computer science or related field.
• 8+ years of Data Engineering experience with 2+ years in a leadership role.
• Strong SQL and Python skills.
• Experience with Spark, Flink, ETL/ELT frameworks, and modern data architectures.
• Hands-on experience with Microsoft Azure services such as Azure Data Factory, Synapse, Data Lake, and Databricks.
• Experience with Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Power BI integration).
• Experience building data solutions for reporting, analytics, dashboards, and web applications.
• Strong understanding of data modeling, governance, metadata management, and cloud-native architectures.
• Excellent communication and stakeholder management skills.

Preferred Certifications

• Microsoft Fabric Analytics Engineer Associate
• Google Professional Data Engineer
• AWS Certified Data Analytics – Specialty