Data Engineer - Python | PySpark | ETL/ELT

Quest Global; Thiruvananthapuram; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Quest Global in India seeks an experienced backend engineer to design cloud-native microservices that integrate medical systems, PACS, and hospital data, focusing on reliability and security.

You will build data ingestion pipelines (DICOM/HL7), containerize services, and deploy with Kubernetes while contributing to CI/CD and ensuring compliance with healthcare regulations and best practices.

Qualifications

• Bachelor's or Master's degree in CS/SE or related field.
• 5–10 years of backend or cloud software development.
• Python backend microservice development.
• PySpark data processing at scale.
• Microservices REST API design and inter-service communication.
• Data modeling and integration patterns.
• Kubernetes containerized deployment and operations.
• ETL/ELT pipeline design (Airflow/Step Functions).
• SQL proficiency and AWS experience.

Responsibilities

• Design and develop cloud-native backend microservices that collect data from medical systems, hospital information systems, radiology information systems, and PACS.
• Build ingestion pipelines for healthcare and imaging data, including DICOM and HL7 formats.
• Extend platform data-collection frameworks to support new data sources and cloud workflows.
• Develop reliable, scalable, and fault-tolerant data-transport services with retries, error handling, monitoring, and dead-lettering.
• Collaborate with data engineers to ensure ingested data meets schema, integrity and quality standards.
• Package and deploy services using containers and Kubernetes.
• Contribute to automated build, testing, deployment, and CI/CD processes.
• Work with technical leaders to align integration solutions with platform architecture.
• Ensure services comply with cybersecurity, data privacy, and regulatory requirements for healthcare software.
• Develop unit, integration, and end-to-end tests and maintain API/data flow deployment documentation.

Skills

Python

PySpark

Microservices

Data modeling

Kubernetes

ETL/ELT

SQL

AWS

CI/CD

Education

Bachelor's or Master's degree in Computer Science or Software Engineering

Tools

Airflow

Step Functions

Kafka

NATS

RabbitMQ

Job description

Job Requirements

At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a positive difference that contributes to a brighter future. We bring together technologies and industries, alongside the contributions of diverse individuals who are empowered by an intentional workplace culture, to solve problems better and faster.

Key areas of responsibility

• Design and develop cloud-native backend microservices that collect data from medical systems, hospital information systems, radiology information systems, and PACS.
• Build ingestion pipelines for healthcare and imaging data, including DICOM and HL7 formats.
• Extend platform data-collection frameworks to support new data sources, integrations, and cloud-based workflows.
• Develop reliable, scalable, and fault-tolerant data-transport services with appropriate retry, error-handling, monitoring, and dead-letter mechanisms.
• Collaborate with data engineers to ensure ingested data meets defined schema, integrity, and quality standards.
• Package and deploy services using containers and Kubernetes.
• Contribute to automated build, testing, deployment, and CI/CD processes.
• Work with technical leaders and system engineers to align integration solutions with the wider platform architecture.
• Ensure services comply with cybersecurity, data-privacy, and regulatory requirements for healthcare and medical-device software.
• Develop unit, integration, and end-to-end tests and maintain clear documentation for APIs, data flows, and deployment configurations.

We are known for our extraordinary people who make the impossible possible every day. Questians are driven by hunger, humility, and aspiration. We believe that our company culture is the key to our ability to make a true difference in every industry we reach. Our teams regularly invest time and dedicated effort into internal culture work, ensuring that all voices are heard.

We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas. We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds. We shine the brightest when we tap into the many dimensions that thrive across over 21,000 difference-makers in our workplace.

Work Experience

Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.

5–10 years of experience in backend or cloud software development.

Must Have

• Python — backend microservice development
• PySpark — data processing at scale
• Microservices — REST API design and inter-service communication
• Data modelling — understanding of data schemas and integration patterns
• Kubernetes — containerized service deployment and operations
• ETL/ELT – Pipeline design and orchestration (e.g., Airflow, Step Functions, or equivalent)
• SQL
• AWS

Nice To Have

• Healthcare data standards: HL7, DICOM
• Clinical/IGT analytics platforms experience
• Streaming data technologies: Kafka, Kinesis, NATS

Experience implementing CI/CD pipelines and infrastructure-as-code.

Familiarity with message brokers (e.g., Kafka, NATS, RabbitMQ) for event-driven data ingestion.

Understanding of data privacy regulations and secure data handling practices in healthcare.

Good collaboration skills in cross-functional agile teams.

Proficiency in leveraging AI-assisted development tools to accelerate coding and testing.