Azure Data Engineer

Persistent Systems Limited; Pune District; On-site; INR 1,500,000 - 2,800,000; Full time

Benefits offered by this job

Hybrid work model

Education sponsorship

Insurance coverage: group term life, 0

Job summary

Persistent Systems Limited in India is seeking an experienced Azure Data Engineer to design, develop, and optimize enterprise-scale data pipelines on Azure. You will work across ADF, Databricks, PySpark, SQL, and Python to deliver analytics-ready data solutions for business-critical initiatives.

The role emphasizes scalable ETL/ELT frameworks, data quality, and collaboration with analysts and data architects. Exposure to healthcare data processing and modern lakehouse architectures is a plus.

Qualifications

• 5–8 years of data engineering and cloud data platform experience.
• Proficient in Azure Data Factory and Azure Databricks.
• Strong PySpark, Python, and SQL skills.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines with ADF and ADB.
• Build large-scale data processing with PySpark, Spark SQL, Python, and SQL.
• Translate business requirements into scalable data solutions.
• Manage data ingestion, transformation, validation, orchestration, and monitoring.
• Collaborate with analysts, architects, and stakeholders on analytics needs.
• Contribute to CI/CD and deployment automation.

Skills

Azure Data Factory

Azure Databricks

PySpark

Python

SQL

Snowflake

Airflow

GitHub Actions

DevOps

Tools

Apache Airflow

GitHub Actions

Snowflake

Job description

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Data Factory (ADF), Azure Databricks (ADB), PySpark, SQL, and Python. The ideal candidate will be responsible for designing, developing, and optimizing enterprise-scale data pipelines and cloud-based data solutions that support analytics, reporting, and business-critical data initiatives. You will play a key role in building scalable ETL/ELT frameworks, enabling enterprise data transformation, and delivering high-quality data solutions across Azure-based platforms. Experience in healthcare data processing, cloud-native architectures, and modern data engineering practices will be an added advantage.

• Location: All Persistent Locations
• Experience: 5 to 8 Years
• Job Type: Full-Time Employment

What You'll Do:

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks (ADB).
• Develop and optimize large-scale data processing solutions using PySpark, Spark SQL, Python, and SQL.
• Gather, analyze, and translate business requirements into scalable technical data solutions.
• Build and manage data ingestion, transformation, validation, orchestration, and monitoring workflows.
• Design reusable frameworks for data integration, transformation, and data quality validation.
• Implement scalable data processing solutions to support reporting, analytics, and AI/ML initiatives.
• Ensure data quality, integrity, availability, reliability, and performance across enterprise data platforms.
• Collaborate with business analysts, data architects, reporting teams, and stakeholders to support analytics and reporting requirements.
• Troubleshoot production issues, resolve data pipeline bottlenecks, and implement performance improvements.
• Participate in code reviews, testing, deployment, release management, and production support activities.
• Implement cloud security, data governance, compliance, and operational best practices.
• Support end-to-end ownership of operational responsibilities to ensure platform stability and successful delivery of critical business initiatives.
• Contribute to CI/CD implementations and deployment automation initiatives.

Expertise You'll Bring:

• 5-8 years of experience in Data Engineering and Cloud Data Platform development.
• Strong hands-on expertise in Azure Data Factory (ADF) and Azure Databricks (ADB).
• Experience building scalable ETL/ELT pipelines and enterprise data integration frameworks.
• Strong proficiency in PySpark, Spark SQL, Python, and SQL for large-scale data processing.
• Experience with Snowflake data warehousing and cloud-based analytics platforms.
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Strong understanding of data warehousing concepts, dimensional modeling, and modern Lakehouse architectures.
• Experience implementing CI/CD pipelines using GitHub Actions and DevOps best practices.
• Strong SQL development skills including query optimization, troubleshooting, and performance tuning.
• Experience building scalable cloud-native data solutions and supporting enterprise analytics environments.
• Knowledge of data governance, monitoring, MLOps, and AI lifecycle management.
• Understanding of data security, compliance requirements, and cloud best practices.
• Exposure to healthcare data processing and healthcare analytics platforms is preferred.
• Strong analytical, troubleshooting, problem-solving, and communication skills.
• Ability to work independently and collaborate effectively within cross-functional teams.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."