Lead Synapse Azure Data Engineer

EXL; Pune District, Bengaluru, Delhi; Hybrid; INR 3,000,000 - 5,500,000; Full time

Job summary

EXL is seeking a Lead Synapse Data Engineer to drive design, development, deployment, and support of enterprise-scale data platforms using Azure Synapse Analytics, ADF, ADLS Gen2, and SQL Server. You will mentor engineers, establish best practices, and lead modernization initiatives across BI, analytics, AI, and data science use cases.

The role requires strong cloud data architecture expertise, Medallion Architecture, Data Vault, and robust governance and data quality practices.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
• 10+ years of experience in Data Engineering, Data Integration, Data Warehousing, and Cloud Data Platforms.
• 5+ years of hands-on experience with Azure Synapse Analytics.
• Strong expertise in Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, and SQL Server.
• Extensive experience designing and implementing enterprise-scale ETL/ELT solutions.
• Strong hands-on expertise with PySpark, Spark SQL, Python, and T-SQL.
• Experience implementing Data Vault, Dimensional Modeling, ODS, and Data Warehouse architectures.
• Strong understanding of Medallion Architecture and modern Lakehouse patterns.
• Experience working with metadata-driven and parameterized data engineering frameworks.
• Expertise in source-to-target mappings, data lineage, data quality, and governance practices.
• Strong background in Agile delivery methodologies and Azure DevOps.
• Experience with Git-based version control and CI/CD implementation.
• Excellent stakeholder management, communication, leadership, and problem-solving skills.

Responsibilities

• Lead the design, development, and implementation of enterprise-scale data solutions using Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Azure SQL technologies.
• Design and develop scalable batch, micro-batch, and near real-time data ingestion and processing pipelines across cloud and on-premises data sources.
• Build and optimize ELT/ETL frameworks leveraging Synapse Pipelines, Dedicated SQL Pools, Serverless SQL Pools, Spark Pools, PySpark, and SQL-based transformations.
• Drive the implementation of modern data platform architectures, including Data Lakehouse, Medallion Architecture, and enterprise data warehouse solutions.
• Design and maintain reusable metadata-driven data integration frameworks.
• Develop robust ingestion mechanisms for databases, APIs, SaaS apps, streaming sources, and flat-files.
• Implement incremental loading, CDC, audit controls, reconciliation, and error-handling.
• Optimize transformation workloads for performance, scalability, reliability, and cost efficiency.

Skills

Leadership

Stakeholder management

Problem-solving

Communication

Education

Bachelor's degree in Computer Science/Engineering/IS

Tools

Azure Synapse Analytics

Azure Data Factory

Azure Data Lake Storage

Azure SQL Database

SQL Server

PySpark

Spark SQL

Python

T-SQL

Azure DevOps

GitHub

Databricks

Job description

Job Title: Lead Synapse Data Engineer

Job Summary

The Lead Synapse Data Engineer is responsible for leading the design, development, deployment, and support of enterprise-scale data platforms and analytics solutions using Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage, SQL Server, and related Azure services. This role provides technical leadership across data integration, data warehousing, lakehouse implementations, data modeling, and advanced analytics initiatives.

The individual will work closely with business stakeholders, architects, data modelers, analysts, and engineering teams to build scalable and governed data solutions supporting reporting, business intelligence, analytics, AI, and data science use cases. The role also includes mentoring and guiding data engineers, establishing engineering best practices, driving platform modernization initiatives, and ensuring the successful delivery of data programs.

The ideal candidate possesses strong expertise in modern cloud data architectures, Medallion Architecture, Data Vault, dimensional modeling, ETL/ELT frameworks, data governance, and Azure-based analytics platforms.

Key Responsibilities

Azure Synapse & Data Platform Engineering

• Lead the design, development, and implementation of enterprise-scale data solutions using Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Azure SQL technologies.
• Design and develop scalable batch, micro-batch, and near real-time data ingestion and processing pipelines across cloud and on-premises data sources.
• Build and optimize ELT/ETL frameworks leveraging Synapse Pipelines, Dedicated SQL Pools, Serverless SQL Pools, Spark Pools, PySpark, and SQL-based transformations.
• Drive the implementation of modern data platform architectures, including Data Lakehouse, Medallion Architecture, and enterprise data warehouse solutions.

Data Integration & Transformation

• Design and maintain reusable, metadata-driven, and parameterized data integration frameworks.
• Develop robust ingestion mechanisms for databases, APIs, SaaS applications, streaming sources, and flat-file data feeds.
• Implement incremental loading, Change Data Capture (CDC), audit controls, reconciliation processes, and error-handling mechanisms.
• Optimize transformation workloads for performance, scalability, reliability, and cost efficiency.

Data Warehouse & Data Modeling

• Collaborate with Data Architects and Data Modelers to implement Conceptual, Logical, and Physical Data Models.
• Understand Data Vault, Dimensional Models, Operational Data Stores (ODS), and Data Marts.
• Build fact and dimension tables to support enterprise reporting, self-service analytics, and business intelligence initiatives.
• Validate data transformations against source-to-target mappings and business requirements.

Data Quality & Governance

• Implement data quality frameworks, validation rules, reconciliation checks, and monitoring controls.
• Support metadata management, data lineage, business glossary integration, and governance initiatives.
• Ensure compliance with enterprise data standards, security controls, privacy requirements, and audit policies.
• Support data contract implementation and enforcement across producer and consumer teams.

Performance Optimization

• Optimize Synapse workloads through partitioning, indexing, file sizing, workload management, and query tuning.
• Analyze and improve SQL performance, Spark execution plans, and pipeline runtimes.
• Monitor platform health, capacity utilization, and operational KPIs.
• Drive cost optimization initiatives across Synapse and Azure consumption services.

Leadership & Delivery

• Lead and mentor teams of Data Engineers, promoting engineering excellence and best practices.
• Conduct code reviews, design reviews, and technical walkthroughs.
• Support project planning, estimation, resource allocation, and delivery governance activities.
• Collaborate with Project Managers, Product Owners, Architects, and stakeholders to ensure successful project delivery.
• Contribute to CoE initiatives, reusable accelerators, frameworks, standards, and innovation programs.

DevOps & Automation

• Implement CI/CD pipelines using Azure DevOps, GitHub, and infrastructure automation tools.
• Support automated testing, deployment, release management, and environment promotion processes.
• Establish monitoring, logging, alerting, and operational support frameworks.
• Develop reusable deployment and operational automation solutions.

Analytics & Reporting Enablement

• Support downstream reporting and analytics solutions by delivering governed and curated datasets.
• Collaborate with BI teams to enable Power BI, Tableau, and other reporting platforms.

Required Qualifications

• Bachelor's Degree in Computer Science, Engineering, Information Systems, or related discipline.
• 10+ years of experience in Data Engineering, Data Integration, Data Warehousing, and Cloud Data Platforms.
• 5+ years of hands-on experience with Azure Synapse Analytics.
• Strong expertise in Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, and SQL Server.
• Extensive experience designing and implementing enterprise-scale ETL/ELT solutions.
• Strong hands-on expertise with PySpark, Spark SQL, Python, and T-SQL.
• Experience implementing Data Vault, Dimensional Modeling, ODS, and Data Warehouse architectures.
• Strong understanding of Medallion Architecture and modern Lakehouse patterns.
• Experience working with metadata-driven and parameterized data engineering frameworks.
• Expertise in source-to-target mappings, data lineage, data quality, and governance practices.
• Strong background in Agile delivery methodologies and Azure DevOps.
• Experience with Git-based version control and CI/CD implementation.
• Excellent stakeholder management, communication, leadership, and problem-solving skills.

Keywords

Azure Synapse Analytics, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), SQL Server, Azure SQL Database, PySpark, Spark SQL, Python, T-SQL, Azure DevOps, GitHub, Databricks