Data Engineer

Persistent Systems Limited; Pune District; Hybrid; INR 1,400,000 - 2,200,000; Full time

Benefits offered by this job

Hybrid work

Flexible hours

Education sponsorship

Long service awards

Job summary

Persistent Systems Limited is seeking a Data Engineer to design, develop, and optimize enterprise-scale data platforms supporting BI, advanced analytics, AI/ML, and GenAI initiatives.

The role requires strong Azure, Databricks, SQL, and Python skills, with experience in data warehouses, data lakes, and data quality. The candidate will collaborate across analytics, AI/ML, and business teams to deliver trusted data products.

Qualifications

• 5–8 years of Data Engineering experience in enterprise-scale platforms.
• Strong SQL and Python for data processing and ETL.
• Hands-on Azure and Databricks with cloud data platforms.
• Experience with data governance, quality, and security.

Responsibilities

• Design, build, and maintain scalable data pipelines.
• Develop data warehouses and data lakes on cloud.
• Implement ETL/ELT pipelines for large datasets.
• Support AI/ML and GenAI initiatives via data prep.
• Collaborate with BI, analytics, AI/ML, and business teams.
• Ensure data quality, governance, security, and availability.
• Implement DevOps, CI/CD, and IaC for data platforms.
• Triage production data pipeline issues and optimize.

Skills

Azure

Databricks

SQL

Python

LLMs

Generative AI

Tools

Azure Data Factory

Synapse Analytics

Snowflake

Spark

Job description

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.

We are seeking a highly skilled Data Engineer with strong expertise in Azure, Databricks, SQL, Python, Large Language Models (LLMs), and Generative AI technologies. The ideal candidate will be responsible for designing, developing, and optimizing enterprise-scale data platforms that support Business Intelligence, Advanced Analytics, AI/ML, and GenAI-driven initiatives. You will work across the entire data lifecycle, from data ingestion and transformation to analytics enablement and AI-ready data platform development. This role requires strong cloud, big data, and data engineering expertise, along with the ability to collaborate with cross-functional teams to deliver scalable and reliable data solutions.

• Location: All Persistent Locations
• Experience: 5 to 8 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable data pipelines and enterprise data integration frameworks.
• Build and optimize data warehouses, data lakes, and modern cloud-based data platforms.
• Develop and support ETL/ELT pipelines for ingesting, transforming, and processing large-scale datasets.
• Implement data engineering solutions using Azure, Databricks, Python, SQL, and Spark technologies.
• Support AI/ML and Generative AI initiatives through feature engineering, data preparation, and scalable data platform development.
• Collaborate with analytics, business intelligence, AI/ML, and business teams to deliver trusted data products.
• Ensure data quality, governance, availability, security, and performance across enterprise platforms.
• Implement automation, monitoring, observability, and operational excellence practices.
• Optimize data processing workloads and improve data platform scalability and efficiency.
• Participate in architecture reviews and contribute to data platform modernization initiatives.
• Implement DevOps, CI/CD, and Infrastructure as Code practices within data engineering workflows.
• Support production environments and troubleshoot data pipeline and platform issues.
• Work closely with stakeholders to understand business requirements and translate them into scalable technical solutions.

Expertise You'll Bring:

• 5-8 years of experience in Data Engineering, Data Warehousing, or Big Data development.
• Strong expertise in SQL and Python for data engineering and data processing.
• Hands-on experience with Azure cloud platform and Azure Databricks.
• Experience developing and supporting scalable ETL/ELT pipelines.
• Strong understanding of Spark, Data Lake architectures, and cloud-native data platforms.
• Experience with data modeling, data governance, data quality, and performance optimization.
• Experience supporting AI/ML initiatives through feature engineering and data pipeline development.
• Exposure to Large Language Models (LLMs), Generative AI solutions, and AI-enabled analytics platforms.
• Experience with Azure Data Factory, Synapse Analytics, Snowflake, or similar enterprise data platforms.
• Knowledge of CI/CD pipelines, DevOps practices, Infrastructure as Code, and data platform automation.
• Experience working in Agile development environments.
• Understanding of data security, privacy, compliance, and governance best practices.
• Experience working with large-scale structured and unstructured datasets.
• Healthcare, payer/provider, claims, operations, or contact center domain experience is preferred.
• Strong analytical, troubleshooting, communication, and stakeholder management skills.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”