Azure Data Engineer

Persistent Systems; Pune District; Hybrid; INR 1,400,000 - 2,200,000; Full time

Benefits offered by this job

Hybrid work

Education sponsorship

Long service awards

Employee engagement

Job summary

Persistent Systems is seeking an Azure Data Engineer to design and build scalable data platforms on Azure. You will create robust data pipelines, integrate Generative AI capabilities, and ensure performance, data quality, and security across the Azure ecosystem.

The role emphasizes collaboration with cross-functional teams to deliver reliable analytics solutions, with a focus on Delta Lake architectures and cloud-native data processing. Hybrid work model may apply in our India centers.

Qualifications

• 4–8 years of experience in data engineering.
• Strong hands-on experience with Azure Databricks, Apache Spark, PySpark, and Delta Lake.
• Advanced proficiency in Python for data processing, APIs, and automation.
• Strong SQL skills and experience with query optimization.
• Experience with Azure Data Factory, ADLS, and Synapse.
• Familiarity with Git, CI/CD practices, and Azure DevOps.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure Databricks and Python.
• Build and maintain robust ETL/ELT workflows for batch and streaming data processing.
• Develop reusable Python frameworks and Databricks notebooks for ingestion, transformation, and validation.
• Process and transform large-scale datasets using Spark and PySpark.
• Design and implement Delta Lake and medallion architecture for reliability.
• Integrate data solutions with Azure services like ADF, ADLS, and Synapse.
• Implement data quality, governance, security, and monitoring mechanisms.
• Collaborate with business and engineering teams to translate requirements into production-ready solutions.

Skills

Azure Data Engineering

Azure Databricks

Python

Generative AI

SQL

Spark/PySpark

Data pipelines

Tools

Git

Azure DevOps

CI/CD

Databricks notebooks

Apache Spark

Job description

We are looking for an Azure Data Engineer with 4 to 8 years of experience and strong expertise in Azure Databricks, Python, and Generative AI. The candidate will design and build scalable data platforms and AI-enabled data solutions on Azure for analytics and business use cases. The ideal candidate will collaborate with cross-functional teams to develop reliable data pipelines, integrate Generative AI capabilities, and ensure high standards of performance, data quality, security, and scalability across the Azure ecosystem.

• Experience: 4 to 8 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and optimize scalable data pipelines using Azure Databricks and Python
• Build and maintain robust ETL/ELT workflows for batch and streaming data processing
• Develop reusable Python frameworks and Databricks notebooks for data ingestion, transformation, and validation
• Process and transform large-scale datasets using Apache Spark and PySpark
• Design and implement Delta Lake and medallion architecture for reliable data processing
• Integrate data solutions with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
• Implement data quality, governance, security, monitoring, and performance-tuning mechanisms
• Collaborate with business, analytics, and engineering teams to translate requirements into production-ready solutions
• Troubleshoot and resolve data pipeline, Databricks, Python, and production issues

Expertise You'll Bring:

• 4 to 8 years of experience in Data Engineering
• Strong hands-on experience with Azure Databricks, Apache Spark, PySpark, and Delta Lake
• Advanced proficiency in Python, including data processing, APIs, reusable modules, and automation
• Strong proficiency in SQL and experience with query optimization
• Experience with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
• Strong understanding of data warehousing, ETL/ELT, data modeling, medallion architecture, and pipeline design
• Working knowledge of Git, CI/CD practices, and Azure DevOps
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”