Senior Azure Data Engineer

Persistent Systems Limited; Pune District; Hybrid; INR 3,500,000 - 5,500,000; Full time

Benefits offered by this job

Competitive salary

Education stipend

Hybrid work model

Cutting-edge technologies

Long Service awards

Insurance coverage

Job summary

Persistent Systems Limited is seeking an experienced Senior Azure Data Engineer to design and optimize cloud-based data platforms supporting enterprise analytics. You will build scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Snowflake, ensuring data quality and governance across lakehouse architectures.

The role requires 6-10 years of data engineering experience, strong SQL, and hands-on Azure Cloud expertise.

Qualifications

• 6-10 years of experience in Data Engineering and Cloud Data Platform development.
• Strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Proven experience with Snowflake including data loading, data transformation, query optimization, data sharing, and performance tuning.
• Strong SQL expertise including complex query development, optimization, troubleshooting, and performance management.
• Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps.
• Good understanding of Data Warehousing principles, Dimensional Modeling, ETL/ELT frameworks, and Lakehouse architecture.
• Experience building and supporting enterprise-scale data pipelines and analytics platforms.
• Knowledge of data governance, data quality, metadata management, and security best practices.
• Experience with Git, CI/CD pipelines, source control management, and Agile methodologies.
• Strong understanding of distributed computing and cloud-native data architectures.
• Strong analytical, troubleshooting, debugging, and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Ability to work independently while collaborating effectively across distributed teams.
• Competitive salary and benefits package

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build and optimize ETL/ELT workflows for processing structured, semi-structured, and unstructured datasets.
• Develop and manage Snowflake data warehouse solutions including data ingestion, transformation, and performance optimization.
• Create reusable frameworks for data ingestion, orchestration, transformation, validation, and monitoring.
• Integrate data from various enterprise applications, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake environments.
• Design and implement modern data lake and Lakehouse architectures.
• Optimize data processing workloads for performance, reliability, scalability, and cost efficiency.
• Implement data quality, governance, security, and monitoring best practices.
• Collaborate with business stakeholders, solution architects, data analysts, and engineering teams to deliver data-driven solutions.
• Troubleshoot and resolve production issues related to data pipelines and cloud data platforms.
• Participate in code reviews, CI/CD implementations, deployment automation, and DevOps initiatives.
• Support data platform modernization efforts and drive continuous improvement initiatives.
• Contribute to architecture discussions and recommend best practices for scalable cloud data solutions.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Snowflake

SQL

Azure Data Factory

ADLS Gen2

Azure DevOps

Lakehouse

Tools

Git

CI/CD

Job description

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.

We are looking for an experienced Senior Azure Data Engineer with strong expertise in Azure Databricks, Snowflake, PySpark, and SQL. The ideal candidate will be responsible for designing, developing, and optimizing scalable cloud-based data platforms that support enterprise analytics, business intelligence, and digital transformation initiatives. You will play a key role in building modern data architectures, developing enterprise-grade ETL/ELT pipelines, and enabling data-driven decision-making across the organization. The role requires hands-on experience with Azure cloud services, Snowflake data warehousing, and large-scale data processing frameworks.

• Experience: 6 to 10 years
• Job Type: Full Time Employment

What You’ll Do:

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build and optimize ETL/ELT workflows for processing structured, semi-structured, and unstructured datasets.
• Develop and manage Snowflake data warehouse solutions including data ingestion, transformation, and performance optimization.
• Create reusable frameworks for data ingestion, orchestration, transformation, validation, and monitoring.
• Integrate data from various enterprise applications, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake environments.
• Design and implement modern data lake and Lakehouse architectures.
• Optimize data processing workloads for performance, reliability, scalability, and cost efficiency.
• Implement data quality, governance, security, and monitoring best practices.
• Collaborate with business stakeholders, solution architects, data analysts, and engineering teams to deliver data-driven solutions.
• Troubleshoot and resolve production issues related to data pipelines and cloud data platforms.
• Participate in code reviews, CI/CD implementations, deployment automation, and DevOps initiatives.
• Support data platform modernization efforts and drive continuous improvement initiatives.
• Contribute to architecture discussions and recommend best practices for scalable cloud data solutions.

Expertise You’ll Bring:

• 6-10 years of experience in Data Engineering and Cloud Data Platform development.
• Strong hands‑on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Proven experience with Snowflake including data loading, data transformation, query optimization, data sharing, and performance tuning.
• Strong SQL expertise including complex query development, optimization, troubleshooting, and performance management.
• Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps.
• Good understanding of Data Warehousing principles, Dimensional Modeling, ETL/ELT frameworks, and Lakehouse architecture.
• Experience building and supporting enterprise‑scale data pipelines and analytics platforms.
• Knowledge of data governance, data quality, metadata management, and security best practices.
• Experience with Git, CI/CD pipelines, source control management, and Agile methodologies.
• Strong understanding of distributed computing and cloud‑native data architectures.
• Strong analytical, troubleshooting, debugging, and problem‑solving skills.
• Excellent communication and stakeholder management abilities.
• Ability to work independently while collaborating effectively across distributed teams.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting‑edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility‑friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”