Azure Data Engineer Lead

Softview Infotech; Chennai District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Softview Infotech is seeking a hands-on Azure Data Engineer Lead to architect, build, and scale enterprise-grade data platforms on Microsoft Azure. You will lead a team of data engineers and partner with architects, analysts, and stakeholders to transform raw data into reliable, high-performance pipelines that power analytics, reporting, and AI initiatives.

Responsibilities include designing scalable pipelines with ADF, Databricks, and Synapse, building lakehouse solutions, enforcing CI/CD, and

Qualifications

• Hands-on Azure Data Engineer Lead with enterprise-scale experience.
• Experience architecting data pipelines on Azure data services.
• Ability lead a data engineering team and define best practices.

Responsibilities

• Lead design and optimization of scalable data pipelines using ADF, Databricks, and Synapse.
• Architect data lakehouse solutions with ADLS Gen2, Delta Lake, Synapse.
• Enforce coding standards and CI/CD across the team.
• Own end-to-end data architecture: ingestion, transformation, orchestration, governance.
• Collaborate with architects, BI, and stakeholders.
• Optimize pipeline performance and costs at scale.
• Implement data quality, security, lineage, and monitoring.
• Mentor data engineers and conduct design reviews.
• Drive Azure Well-Architected Framework adherence.
• Stay updated with emerging Azure data services.

Skills

Azure Data Factory

Azure Databricks

Azure Synapse

ADLS Gen2

PySpark

SQL

Python

Delta Lake

Lakehouse

CI/CD

Terraform

RBAC

Purview

Encryption

Agile

Stakeholder mgmt

Communication

Tools

Terraform

ARM/Bicep

Job description

Location: Chennai, Bangalore

Experience: 8–10 Years

Location: Chennai, Bangalore

Key Skills: Azure Data Engineering, Azure Databricks, Azure Data Factory, PySpark, SQL

About The Role

We are looking for a hands-on Azure Data Engineer Lead to architect, build, and scale enterprise-grade data platforms on Microsoft Azure. You will lead a team of data engineers, define best practices, and partner with architects, analysts, and business stakeholders to turn raw data into reliable, high-performance pipelines that power analytics, reporting, and AI initiatives.

Key Responsibilities

• Lead the design, development, and optimization of scalable data pipelines using Azure Data Factory, Azure Databricks, and Synapse Analytics
• Architect modern data lakehouse solutions using ADLS Gen2, Delta Lake, and Synapse/Fabric
• Define and enforce data engineering standards, coding guidelines, and CI/CD practices across the team
• Own end-to-end data architecture — ingestion, transformation, orchestration, and governance
• Collaborate with data architects, BI teams, and business stakeholders to translate requirements into scalable technical solutions
• Optimize pipeline performance, cost, and reliability across large-scale distributed datasets
• Implement robust data quality, security, lineage, and monitoring frameworks (Purview, Key Vault, Log Analytics)
• Mentor and guide a team of data engineers; conduct code reviews and technical design sessions
• Drive cloud cost optimization and adherence to Azure Well-Architected Framework principles
• Stay current with emerging Azure data services and evaluate their adoption

Required Skills

• Strong hands-on expertise in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, ADLS Gen2
• Proficiency in PySpark, SQL, and Python for large-scale data processing
• Experience with Delta Lake / Lakehouse architecture
• Strong understanding of data modeling (dimensional, star/snowflake schema)
• Experience with CI/CD pipelines (Azure DevOps) and Infrastructure-as-Code (Terraform/ARM/Bicep)
• Solid grasp of data governance, security, and compliance (RBAC, Azure Purview, encryption)
• Experience leading teams and managing project delivery in an Agile environment
• Strong communication and stakeholder management skills

Good To Have

• Experience with Microsoft Fabric / Power BI
• Exposure to Azure Event Hub / Stream Analytics for real-time processing
• Knowledge of MLOps or integration with Azure Machine Learning
• Microsoft certifications: DP-203 (Azure Data Engineer Associate), Azure Solutions Architect

Why Join Us?

• Work on cutting-edge cloud data platforms at enterprise scale
• Lead a talented team and shape data engineering practices from the ground up
• Competitive compensation, flexible work model, and continuous learning opportunities

Skills: azure data factory,sql,pyspark,azure databricks,azure data engineer,azure data engineering