Azure Data Engineer - Databricks & Snowflake

Persistent Systems Limited; Pune District; Hybrid; INR 4,500,000 - 7,000,000; Full time

Benefits offered by this job

Group term life insurance

Personal accident insurance

Mediclaim hospitalization

Hybrid work model

Higher education sponsorship

Long service awards

Flexible work hours

Job summary

Persistent Systems Limited is seeking an experienced Azure Data Engineer in Pune to design, develop, and optimize modern cloud-based data platforms across Azure and Snowflake ecosystems. The role focuses on building scalable data pipelines, data warehousing, and analytics solutions with Delta Lake architectures.

The candidate should have 8-12 years of data engineering experience and hands-on expertise in Azure Databricks, PySpark, Snowflake, and SQL, along with strong governance and security

Qualifications

• 8-12 years of experience in Data Engineering and Cloud Data Platform development.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build, optimize, and support ETL/ELT workflows for large data volumes.
• Develop Snowflake data warehouse solutions including data modeling and transformations.
• Create reusable frameworks for data ingestion, transformation, orchestration, and validation.
• Integrate data from enterprise systems, databases, APIs into Azure Data Lake and Snowflake.
• Implement and maintain Delta Lake architectures to support scalable processing.
• Optimize data processing workloads for performance, scalability, reliability, and cost.
• Collaborate with stakeholders, architects, analysts, data science teams.
• Implement data governance, data quality, security, and monitoring best practices.
• Troubleshoot production issues related to data pipelines and cloud platforms.
• Participate in code reviews, automation, CI/CD, deployment activities.
• Support data platform modernization and continuous improvement.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

SQL

Azure Data Factory

Azure Data Lake Storage

Azure Key Vault

Azure DevOps

Data Warehousing

ETL/ELT

Data Governance

Tools

Snowflake

Git

CI/CD

Job description

Job Description

About Persistent

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Databricks, Snowflake, PySpark, and SQL to design, develop, and optimize modern cloud-based data platforms. The ideal candidate will play a critical role in building scalable data pipelines, implementing enterprise data warehousing solutions, and enabling advanced analytics and business intelligence initiatives across Azure and Snowflake ecosystems.

• Experience: 3 to 8 Years
• Job Type: Full-Time Employment

What You'll Do:

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build, optimize, and support ETL/ELT workflows for processing large volumes of structured and unstructured data.
• Develop and manage Snowflake data warehouse solutions, including data modeling, transformation, and performance optimization.
• Create reusable frameworks for data ingestion, data transformation, orchestration, and data validation.
• Integrate data from various enterprise systems, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake environments.
• Implement and maintain Delta Lake architectures to support scalable and reliable data processing.
• Optimize data processing workloads for performance, scalability, reliability, and cost efficiency.
• Collaborate with business stakeholders, architects, analysts, and data science teams to deliver business-driven data solutions.
• Implement data governance, data quality, security, and monitoring best practices.
• Troubleshoot production issues related to data pipelines, Snowflake environments, and cloud data platforms.
• Participate in code reviews, automation initiatives, CI/CD implementations, and deployment activities.
• Support data platform modernization initiatives and continuous improvement programs.

Expertise You'll Bring:

• 8-12 years of experience in Data Engineering and Cloud Data Platform development.
• Strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Proven experience with Snowflake including data loading, transformations, data sharing, and performance optimization.
• Strong proficiency in SQL for complex query development, troubleshooting, and performance tuning.
• Experience with Azure services such as Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps.
• Strong understanding of data warehousing concepts, dimensional modeling, ETL/ELT architecture, and Lakehouse architecture.
• Experience developing and supporting enterprise-scale data pipelines and analytics platforms.
• Knowledge of data governance, data quality management, and cloud security best practices.
• Experience with Git, CI/CD pipelines, DevOps practices, and Agile delivery methodologies.
• Strong analytical, debugging, troubleshooting, and problem-solving skills.
• Ability to work independently and collaborate effectively across distributed teams.
• Excellent communication and stakeholder management skills.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."