Data Engineer + GenAI

Persistent; Pune District; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Competitive salary package

Career growth opportunities with教育‑s支

Job summary

Persistent is seeking a Senior Data Engineer to design and optimize scalable data platforms enabling analytics, AI, and BI initiatives. You will lead batch and real-time processing using Spark, Kafka, and AWS services, while ensuring data quality, governance, and healthcare interoperability with FHIR standards.

You will collaborate with stakeholders to translate business requirements into robust data solutions, mentor junior engineers, and contribute to engineering excellence in a hybrid

Qualifications

• 8 to 12 years of experience in Data Engineering, Data Warehousing, and Big Data technologies.
• Strong hands-on experience with Python, SQL, Spark, and Databricks.
• Experience designing enterprise-scale ETL/ELT data pipelines and data models.
• Knowledge of AWS services including Glue, Redshift, Athena, and S3.
• Experience with streaming systems like Kafka and Kinesis.
• Understanding of data governance, quality, lineage, and compliance frameworks.
• Familiarity with healthcare data standards, particularly FHIR.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions.
• Build and optimize data models to support analytics, AI workloads, and reporting.
• Develop batch and real-time data processing using Spark and Kafka/Kinesis.
• Work with AWS services to build cloud-native data solutions.
• Implement data governance, quality, and security across data ecosystems.
• Collaborate with stakeholders to translate requirements into technical solutions.
• Mentor junior engineers and promote data engineering best practices.

Skills

Python

SQL

Spark

Databricks

AWS

Data Modeling

Data Warehousing

ETL/ELT

CI/CD for data engineering

FHIR

Tools

Databricks

Spark

Kafka

Kinesis

AWS Glue

Redshift

Athena

S3

GitHub Copilot

Job description

About Position:

We are seeking an experienced Sr. Data Engineer/Data Engineer + GenAI to design, build, and optimize modern data platforms that enable enterprise-scale analytics, reporting, AI, and business intelligence initiatives. The ideal candidate will possess strong expertise in Databricks, Spark, Kafka, Python, SQL, and AWS data services, with the ability to architect scalable batch and real-time data processing solutions. This role requires collaboration with business and technology stakeholders to deliver reliable, secure, and high-quality data solutions while supporting healthcare data interoperability through FHIR standards and leveraging AI-powered engineering practices.

• Role: Senior Data Engineer
• Location: Pune
• Experience: 8 to 12 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions using modern data engineering practices.
• Build and optimize data models to support reporting, analytics, machine learning, and AI workloads.
• Develop and manage large-scale batch and real-time data processing solutions using Spark and Kafka/Kinesis.
• Work extensively with AWS services including Glue, Redshift, Athena, and S3 to build cloud-native data solutions.
• Implement and optimize data integration, transformation, and orchestration workflows.
• Leverage Databricks to build scalable data engineering and analytics platforms.
• Ensure data quality, governance, security, and compliance standards across enterprise data ecosystems.
• Collaborate with business stakeholders, architects, analysts, and developers to translate business requirements into technical solutions.
• Design and implement healthcare data integration and interoperability solutions using FHIR standards.
• Monitor and optimize data pipeline performance, scalability, and reliability.
• Support data migration, modernization, and cloud transformation initiatives.
• Implement best practices for data architecture, metadata management, and data lifecycle management.
• Contribute to code reviews, technical design discussions, and engineering excellence initiatives.
• Mentor junior engineers and promote data engineering best practices across teams.
• Utilize GitHub Copilot, Microsoft 365 Copilot, and approved AI tools to improve engineering productivity and software quality.
• Stay current with emerging AI technologies, data engineering trends, and industry best practices.

Expertise You'll Bring:

• 8 to 12 years of experience in Data Engineering, Data Warehousing, and Big Data technologies.
• Strong hands-on experience with Python, SQL, Spark, and Databricks.
• Expertise in designing and developing enterprise-scale ETL/ELT data pipelines.
• Strong experience in Data Modeling, Data Architecture, and Data Warehousing concepts.
• Hands-on experience with AWS Glue, Redshift, Athena, and S3.
• Expertise in streaming technologies such as Kafka and Kinesis.
• Strong understanding of data governance, data quality, lineage, and compliance frameworks.
• Experience building scalable batch and real-time processing architectures.
• Strong performance tuning and optimization experience for Spark and SQL workloads.
• Experience implementing CI/CD practices for data engineering projects.
• Knowledge of healthcare data standards, particularly FHIR (Fast Healthcare Interoperability Resources).
• Experience supporting analytics, reporting, AI, and machine learning data platforms.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalisation for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment.

Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.