Data Engineer + GenAI

Persistent Systems; Pune District; Hybrid; INR 2,800,000 - 4,000,000; Full time

Benefits offered by this job

Competitive salary

Culture focused on talent development

Cutting-edge technologies

Employee engagement initiatives

Annual health check-ups

Insurance coverage for family

Job summary

Persistent Systems in Pune is seeking an experienced Senior Data Engineer to design, build, and optimize modern data platforms for analytics, reporting, AI, and BI. You will work with Databricks, Spark, Kafka, Python, SQL, and AWS data services to craft scalable batch and real-time solutions that meet healthcare data interoperability needs.

Collaborate with stakeholders to deliver reliable, secure data solutions, implement data governance, and mentor junior engineers while adopting AI-powered

Qualifications

• 8–12 years of experience in Data Engineering, Data Warehousing, and Big Data technologies.
• Hands-on with Python, SQL, Spark, and Databricks.
• Experience designing and developing enterprise-scale ETL/ELT pipelines.
• Experience with AWS Glue, Redshift, Athena and S3.
• Experience with streaming: Kafka and Kinesis; healthcare data standards (FHIR).
• CI/CD practices for data engineering; data governance and quality.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions.
• Build data models to support reporting, analytics, ML, and AI workloads.
• Develop batch and real-time processing using Spark and Kafka/Kinesis.
• Work with AWS services to build cloud-native data solutions.
• Implement data integration, transformation, and orchestration workflows.
• Leverage Databricks for scalable data engineering and analytics platforms.
• Ensure data quality, governance, security, and compliance.
• Collaborate with stakeholders to translate requirements into solutions.
• Design and implement healthcare data integration using FHIR standards.
• Mentor junior engineers and promote best practices.

Skills

Python

SQL

Spark

Databricks

Kafka

Tools

AWS Glue

Redshift

Athena

S3

Kinesis

FHIR

GitHub Copilot

Job description

About Position:

We are seeking an experienced Sr. Data Engineer/Data Engineer + GenAI to design, build, and optimize modern data platforms that enable enterprise-scale analytics, reporting, AI, and business intelligence initiatives. The ideal candidate will possess strong expertise in Databricks, Spark, Kafka, Python, SQL, and AWS data services, with the ability to architect scalable batch and real-time data processing solutions. This role requires collaboration with business and technology stakeholders to deliver reliable, secure, and high-quality data solutions while supporting healthcare data interoperability through FHIR standards and leveraging AI-powered engineering practices.

• Role: Senior Data Engineer
• Location: Pune
• Experience: 8 to 12 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions using modern data engineering practices.
• Build and optimize data models to support reporting, analytics, machine learning, and AI workloads.
• Develop and manage large-scale batch and real-time data processing solutions using Spark and Kafka/Kinesis.
• Work extensively with AWS services including Glue, Redshift, Athena, and S3 to build cloud-native data solutions.
• Implement and optimize data integration, transformation, and orchestration workflows.
• Leverage Databricks to build scalable data engineering and analytics platforms.
• Ensure data quality, governance, security, and compliance standards across enterprise data ecosystems.
• Collaborate with business stakeholders, architects, analysts, and developers to translate business requirements into technical solutions.
• Design and implement healthcare data integration and interoperability solutions using FHIR standards.
• Monitor and optimize data pipeline performance, scalability, and reliability.
• Support data migration, modernization, and cloud transformation initiatives.
• Implement best practices for data architecture, metadata management, and data lifecycle management.
• Contribute to code reviews, technical design discussions, and engineering excellence initiatives.
• Mentor junior engineers and promote data engineering best practices across teams.
• Utilize GitHub Copilot, Microsoft 365 Copilot, and approved AI tools to improve engineering productivity and software quality.
• Stay current with emerging AI technologies, data engineering trends, and industry best practices.

Expertise You'll Bring:

• 8 to 12 years of experience in Data Engineering, Data Warehousing, and Big Data technologies.
• Strong hands-on experience with Python, SQL, Spark, and Databricks.
• Expertise in designing and developing enterprise-scale ETL/ELT data pipelines.
• Strong experience in Data Modeling, Data Architecture, and Data Warehousing concepts.
• Hands-on experience with AWS Glue, Redshift, Athena, and S3.
• Expertise in streaming technologies such as Kafka and Kinesis.
• Strong understanding of data governance, data quality, lineage, and compliance frameworks.
• Experience building scalable batch and real-time processing architectures.
• Strong performance tuning and optimization experience for Spark and SQL workloads.
• Experience implementing CI/CD practices for data engineering projects.
• Knowledge of healthcare data standards, particularly FHIR (Fast Healthcare Interoperability Resources).
• Experience supporting analytics, reporting, AI, and machine learning data platforms.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalisation for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment.

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."