Azure Data Engineer Lead

Softview Infotech; Chennai District; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Softview Infotech in Chennai/Bangalore seeks a hands-on Azure Data Engineer Lead to architect, build, and scale enterprise‑grade data platforms on Microsoft Azure. You will lead a team of data engineers, define best practices, and partner with architects, analysts, and business stakeholders to turn raw data into reliable, high‑performance pipelines that power analytics, reporting, and AI initiatives.

We expect 8–10 years of experience in Azure data engineering, with strong skills in ADF,

Qualifications

• Hands-on expertise with Azure Data Factory, Databricks, Synapse, and ADLS Gen2.
• Proficient in PySpark, SQL and Python for large-scale data processing.
• Experience with Delta Lake / Lakehouse architecture.
• Strong data modeling skills (dimensional, star/snowflake).
• CI/CD pipelines experience (Azure DevOps) and IaC (Terraform/ARM/Bicep).
• Knowledge of data governance, security, and compliance.
• Experience leading teams and delivering in Agile environments.
• Excellent communication and stakeholder management abilities.

Responsibilities

• Lead the design, development, and optimization of scalable data pipelines on Azure.
• Architect modern data lakehouse solutions using ADLS Gen2, Delta Lake and Synapse.
• Define and enforce data engineering standards, coding guidelines, and CI/CD practices.
• Own end‑to‑end data architecture — ingestion, transformation, orchestration, governance.
• Collaborate with data architects, BI teams, and business stakeholders to translate requirements.
• Optimize pipeline performance, cost, and reliability across large datasets.
• Implement data quality, security, lineage, and monitoring frameworks (Purview, Key Vault).
• Mentor and guide a team of data engineers; conduct code reviews and design reviews.
• Drive cloud cost optimization and Azure Well‑Architected adherence.
• Stay current with emerging Azure data services and assess adoption.

Skills

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

ADLS Gen2

PySpark

SQL

Python

Delta Lake

Lakehouse architecture

Data modeling (dimensional)

Agile team leadership

Tools

Azure DevOps

Terraform

ARM/Bicep

Job description

Location: Chennai, Bangalore

• Strong hands-on expertise in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, ADLS Gen2
• Microsoft certifications: DP-203 (Azure Data Engineer Associate), Azure Solutions Architect

Role: Azure Data Engineer Lead

Experience: 8–10 Years

Location: Chennai, Bangalore

Key Skills: Azure Data Engineering, Azure Databricks, Azure Data Factory, PySpark, SQL

About The Role

We are looking for a hands-on Azure Data Engineer Lead to architect, build, and scale enterprise‑grade data platforms on Microsoft Azure. You will lead a team of data engineers, define best practices, and partner with architects, analysts, and business stakeholders to turn raw data into reliable, high‑performance pipelines that power analytics, reporting, and AI initiatives.

Key Responsibilities

• Lead the design, development, and optimization of scalable data pipelines using Azure Data Factory, Azure Databricks, and Synapse Analytics
• Architect modern data lakehouse solutions using ADLS Gen2, Delta Lake, and Synapse/Fabric
• Define and enforce data engineering standards, coding guidelines, and CI/CD practices across the team
• Own end‑to‑end data architecture — ingestion, transformation, orchestration, and governance
• Collaborate with data architects, BI teams, and business stakeholders to translate requirements into scalable technical solutions
• Optimize pipeline performance, cost, and reliability across large‑scale distributed datasets
• Implement robust data quality, security, lineage, and monitoring frameworks (Purview, Key Vault, Log Analytics)
• Mentor and guide a team of data engineers; conduct code reviews and technical design sessions
• Drive cloud cost optimization and adherence to Azure Well‑Architected Framework principles
• Stay current with emerging Azure data services and evaluate their adoption

Required Skills

• Strong hands‑on expertise in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, ADLS Gen2
• Proficiency in PySpark, SQL, and Python for large‑scale data processing
• Experience with Delta Lake / Lakehouse architecture
• Strong understanding of data modeling (dimensional, star/snowflake schema)
• Experience with CI/CD pipelines (Azure DevOps) and Infrastructure‑as‑Code (Terraform/ARM/Bicep)
• Solid grasp of data governance, security, and compliance (RBAC, Azure Purview, encryption)
• Experience leading teams and managing project delivery in an Agile environment
• Strong communication and stakeholder management skills

Good To Have

• Experience with Microsoft Fabric / Power BI
• Exposure to Azure Event Hub / Stream Analytics for real‑time processing
• Knowledge of MLOps or integration with Azure Machine Learning
• Microsoft certifications: DP-203 (Azure Data Engineer Associate), Azure Solutions Architect

Why Join Us?

• Work on cutting‑edge cloud data platforms at enterprise scale
• Lead a talented team and shape data engineering practices from the ground up
• Competitive compensation, flexible work model, and continuous learning opportunities

Skills: azure data factory,azure data engineer,azure data engineering,azure databricks,sql,pyspark