Senior Azure Data Engineer

Persistent; Mumbai; Hybrid; INR 4,200,000 - 6,500,000; Full time

Benefits offered by this job

Competitive compensation

Growth opportunities

Flexible work hours

Long Service awards

Annual health check-ups

Insurance coverage

Job summary

Persistent is seeking a Senior Azure Data Engineer to lead the design, development, and modernization of enterprise-scale data platforms on Azure. The role focuses on ADF, Databricks, real-time processing, Delta Live Tables, and modern data modeling techniques.

The candidate will architect scalable data solutions, implement governance, and drive best practices across complex environments, with hands-on leadership and mentorship responsibilities.

Qualifications

• 9–13 years of experience in Data Engineering, Data Architecture, and Cloud Data Platforms.
• Deep expertise in Azure Data Factory (ADF) and Azure Databricks (ADB).
• Experience building enterprise-scale data platforms and modernization solutions.
• Real-time data processing experience using Azure Stream Analytics and Delta Live Tables (DLT).

Responsibilities

• Design, develop, and maintain enterprise-grade data pipelines using ADF and Databricks.
• Build scalable batch and real-time data processing solutions for analytics and operational workloads.
• Develop ETL/ELT frameworks for structured, semi-structured, and streaming data.
• Implement robust data ingestion, transformation, enrichment, and consumption architectures.
• Design and implement real-time streaming solutions using Azure Stream Analytics and Databricks.
• Build and manage DLT pipelines for high-volume workloads.
• Develop scalable event-driven architectures for data processing.
• Implement CI/CD pipelines using Azure DevOps.

Skills

Azure Data Factory

Azure Databricks

PySpark

Spark SQL

SQL

Data Modeling

Data Vault 2.0

Kimball Dimensional Modeling

Lakehouse Architecture

Databricks Unity Catalog

Microsoft Purview

Tools

Azure DevOps

Delta Live Tables

Fabric Lakehouse

Fabric Pipelines

Job description

About Position:

We are seeking a highly experienced Senior Azure Data Engineer to lead the design, development, and modernization of enterprise-scale data platforms on Azure. The ideal candidate will have deep expertise in Azure Data Factory (ADF), Azure Databricks (ADB), real-time data processing, Delta Live Tables (DLT), and modern data modeling techniques. This role will be responsible for architecting scalable data solutions, enabling real-time analytics, implementing governance frameworks, and driving data engineering best practices across complex enterprise environments. The candidate should bring strong hands-on technical expertise, leadership capabilities, and experience delivering large-scale cloud data transformation programs.

• Role: Senior Azure Data Engineer
• Location: Mumbai
• Experience: 9 to 13 years
• Job Type: Full-Time Employment

What You'll Do:

• Design, develop, and maintain enterprise-grade data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Build scalable batch and real-time data processing solutions for analytics and operational workloads.
• Develop and optimize ETL/ELT frameworks for structured, semi-structured, and streaming data.
• Implement robust data ingestion, transformation, enrichment, and consumption architectures.
• Design and implement real-time streaming solutions using Azure Stream Analytics and Databricks.
• Build and manage Delta Live Tables (DLT) pipelines for high-volume streaming and batch workloads.
• Develop scalable event-driven architectures supporting business-critical data processing requirements.
• Optimize streaming workloads for performance, reliability, and cost efficiency.
• Design enterprise data models using Data Vault 2.0
• Kimball Dimensional Modeling
• Star & Snowflake Schemas
• Define data architecture standards, modeling frameworks, and governance practices.
• Support modern Lakehouse architectures for enterprise analytics and reporting platforms.
• Implement data governance frameworks leveraging Databricks Unity Catalog.
• Support metadata management, lineage tracking, access controls, and auditability requirements.
• Collaborate with governance teams to ensure compliance with enterprise security policies.
• Implement data quality monitoring, validation, and stewardship processes.
• Build and maintain CI/CD pipelines using Azure DevOps.
• Manage source code repositories, branching strategies, release management, and deployment automation.
• Implement Infrastructure as Code (IaC) and automated deployment frameworks.
• Support platform operationalization through monitoring, alerting, and automated recovery mechanisms.
• Work closely with business stakeholders, architects, data scientists, analysts, and engineering teams.
• Provide technical leadership and mentorship to data engineering teams.
• Conduct architecture reviews, code reviews, and design governance sessions.
• Define engineering standards, reusable frameworks, and best practices.
• Drive continuous improvement initiatives across data platforms and engineering processes.

Expertise You'll Bring:

• 9 to 13 years of experience in Data Engineering, Data Architecture, and Cloud Data Platforms.
• Extensive hands-on expertise in Azure Data Factory (ADF) and Azure Databricks (ADB).
• Strong experience building enterprise-scale data platforms and modernization solutions.
• Proven expertise in real-time data processing using Azure Stream Analytics
• Delta Live Tables (DLT)
• Streaming Data Architectures
• Strong proficiency in PySpark
• Spark SQL
• SQL
• Data Engineering Best Practices
• Deep expertise in Data Modeling methodologies including Data Vault 2.0
• Kimball Methodology
• Dimensional Modeling
• Lakehouse Architecture
• Experience designing and implementing scalable data lakes, warehouses, and modern analytics platforms.
• Hands-on knowledge of Microsoft Fabric, including Fabric Lakehouse
• Fabric Warehouse
• Fabric Pipelines
• Strong understanding of Data Governance frameworks and tools including Databricks Unity Catalog
• Microsoft Purview
• Metadata Management
• Data Lineage
• Access Governance
• Experience implementing CI/CD pipelines and DevOps practices using Azure DevOps.
• Expertise in source control management, automated deployments, and Infrastructure as Code.
• Strong knowledge of cloud security, compliance, and data protection best practices.
• Experience optimizing Spark workloads, storage architectures, and cloud resource utilization.
• Strong leadership, stakeholder management, and mentoring capabilities.
• Excellent communication, presentation, and problem-solving skills.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."