Data Engineering Lead

Verdantas; Pune District; Sur place; INR 3 500 000 - 5 500 000; Plein temps

Résumé du poste

Verdantas is seeking a Data Engineering Lead to design scalable, secure data solutions that support AI/ML, reporting, analytics, and web applications. The role emphasizes cloud data platforms, architecture, and building high-performance data pipelines.

You will lead ETL/ELT development, manage data lakes, lakehouses, and data warehouses on Azure and Microsoft Fabric, mentor engineers, collaborate with stakeholders, and drive governance, quality, and cost optimization across projects.

Qualifications

• Bachelor's or Master’s degree in Computer Science or related field.
• 8+ years of Data Engineering experience with 2+ years in a leadership role.
• Strong SQL and Python skills.
• Experience with Spark, Flink, ETL/ELT frameworks, and modern data architectures.
• Hands‑on experience with Microsoft Azure services such as Azure Data Factory, Synapse, Data Lake, and Databricks.
• Experience with Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Power BI integration).
• Experience building data solutions for reporting, analytics, dashboards, and web applications.
• Strong understanding of data modeling, governance, metadata management, and cloud-native architectures.
• Excellent communication and stakeholder management skills.

Responsabilités

• Design and implement batch and real-time data architectures using modern cloud platforms.
• Define data engineering standards, governance, and best practices.
• Evaluate and recommend technologies for data ingestion, processing, storage, and reporting.
• Lead development of ETL/ELT pipelines using Spark, Airflow, dbt, Azure Data Factory, and Microsoft Fabric.
• Build and manage data lakes, lakehouses, data warehouses, and data marts.
• Ensure data quality, lineage, observability, and governance.
• Develop scalable data solutions supporting enterprise reporting, dashboards, and web applications.
• Mentor data engineers and drive technical excellence.
• Collaborate with business stakeholders, analysts, data scientists, and application teams.
• Manage project delivery, timelines, and resource allocation.
• Build and manage cloud-based data platforms on Azure and Microsoft Fabric.
• Implement CI/CD pipelines and Infrastructure as Code (IaC).
• Optimize performance, scalability, and cost efficiency.

Connaissances

SQL

Python

Spark

Flink

ETL/ELT frameworks

Data modeling

Cloud data platforms

Leadership

Formation

Bachelor's or Master's degree in Computer Science or related field

Outils

Azure Data Factory

Synapse

Data Lake

Databricks

Microsoft Fabric

OneLake

Lakehouse

Power BI integration

Description du poste

Verdantas is seeking a Data Engineering Lead to design and deliver scalable, secure, and modern data solutions that support AI/ML, reporting, analytics, and web applications. The ideal candidate will have strong expertise in Microsoft Azure, Microsoft Fabric, data platform architecture, and team leadership.

Key Responsibilities

Architecture & Strategy

• Design and implement batch and real-time data architectures using modern cloud platforms.
• Define data engineering standards, governance, and best practices.
• Evaluate and recommend technologies for data ingestion, processing, storage, and reporting.

Development & Operations

• Lead development of ETL/ELT pipelines using Spark, Airflow, dbt, Azure Data Factory, and Microsoft Fabric.
• Build and manage data lakes, lakehouses, data warehouses, and data marts.
• Ensure data quality, lineage, observability, and governance.
• Develop scalable data solutions supporting enterprise reporting, dashboards, and web applications.
• Mentor data engineers and drive technical excellence.
• Collaborate with business stakeholders, analysts, data scientists, and application teams.
• Manage project delivery, timelines, and resource allocation.
• Build and manage cloud-based data platforms on Azure and Microsoft Fabric.
• Implement CI/CD pipelines and Infrastructure as Code (IaC).
• Optimize performance, scalability, and cost efficiency.

Skills & Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• 8+ years of Data Engineering experience with 2+ years in a leadership role.
• Strong SQL and Python skills.
• Experience with Spark, Flink, ETL/ELT frameworks, and modern data architectures.
• Hands‑on experience with Microsoft Azure services such as Azure Data Factory, Synapse, Data Lake, and Databricks.
• Experience with Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Power BI integration).
• Experience building data solutions for reporting, analytics, dashboards, and web applications.
• Strong understanding of data modeling, governance, metadata management, and cloud-native architectures.
• Excellent communication and stakeholder management skills.

Preferred Certifications

• Microsoft Fabric Analytics Engineer Associate
• Google Professional Data Engineer
• AWS Certified Data Analytics – Specialty