Pyspark Developer

enGen Global; Chennai District, Hyderabad; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

enGen Global is seeking a Data Integration Developer in India to design, develop, and maintain data pipelines for membership, clinical, and claims data. The role emphasizes PySpark proficiency and healthcare data interoperability.

Ideal candidates will have 5–12 years of experience, exposure to AWS/Azure/GCP, and strong analytical skills to ensure data quality and scalable data flows. This position is based in Chennai/Hyderabad, India.

Qualifications

• PySpark: extensive experience in large-scale data processing.
• US Healthcare domain experience is required.
• Experience with HL7 data interoperability is a must.

Responsibilities

• Design, develop, and maintain data integration pipelines for membership, clinical, and claims data.
• Develop PySpark scripts for efficient processing and transformation of large datasets.
• Collaborate with data architects, analysts, and other developers to understand data requirements.

Skills

PySpark

US Healthcare

HL7

Education

Bachelor’s degree

Tools

Informatica

Gen AI

Job description

Job Title: Data Integration Developer

Department: DataWorks, CMS Interoperability Mandate

Location: Chennai/Hyderabad India

Job Summary: We are seeking a highly skilled and motivated Data Integration Developer to join our team.

The ideal candidate will be responsible for designing, developing, and implementing solutions that extract, transform, and load (ETL) membership,

clinical, and claims data from various source systems into a JSON format. This role requires strong expertise in PySpark and healthcare domain experience

to ensure data accuracy, interoperability, and efficient data flow.

Responsibilities:

• Design, develop, and maintain data integration pipelines to extract membership, clinical, and claims data from diverse source systems.
• Develop and optimize PySpark scripts for efficient data processing, transformation, and manipulation of large datasets (structured and non-structured).
• Work closely with data architects, business analysts, and other developers to understand data requirements
• Implement robust data quality checks and validation processes to ensure the integrity and accuracy of data
• Troubleshoot and resolve data integration issues, including data discrepancies, performance bottlenecks, and system errors.
• Participate in the design and implementation of new data models as needed.
• Develop and maintain comprehensive documentation for data integration processes, mappings, and technical specifications.
• Collaborate with cross-functional teams to support testing, deployment, and ongoing maintenance of data integration solutions.

Mandatory/Must Have Skillset:

• PySpark: Extensive experience with PySpark for large-scale data processing
• US Healthcare domain
• Any HL7 work experience

Good to have:

• Informatica
• Gen AI

Education:

Bachelor degree in Computer Science, Information Technology, Data Science, or a related field.

Experience:

• 5-12 years of experience in data integration, ETL development, or software engineering, with a focus on US healthcare data.
• Proven experience working with healthcare data (membership, clinical, claims).
• Experience with cloud platforms (e.g., AWS, Azure, GCP) is a plus.

Soft Skills:

• Excellent analytical and problem-solving skills.
• Strong communication and interpersonal skills, with the ability to collaborate effectively with technical and non-technical stakeholders.
• Ability to work independently and as part of a team in a fast-paced environment.
• Detail-oriented with a commitment to data quality and accuracy.