Senior Data Engineer

GAVS Technologies N.A., Inc; Chennai District; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

GAVS Technologies N.A., Inc is seeking a Senior Data Engineer in Chennai, India. The incumbent will design and optimize data processing systems specifically for healthcare analytics. Responsibilities include developing ETL/ELT pipelines using Python and PySpark, managing data quality, and working with QA and DevOps teams to ensure streamlined operations. Strong skills in AWS Glue and Apache Spark are required. Join a dynamic team dedicated to impactful healthcare solutions.

Qualifications

• Strong knowledge of Python and PySpark.
• Experience with SQL and data pipeline design.
• Familiarity with AWS services like Glue and S3.

Responsibilities

• Design and implement ETL/ELT pipelines using Python and PySpark.
• Develop scalable data workflows with Apache Spark and AWS Glue.
• Collaborate with QA and DevOps teams for CI/CD integration.

Skills

Python

PySpark

SQL

AWS Glue

Apache Spark

Apache Airflow

CI/CD

ETL/ELT pipeline design and optimization

Data modeling and normalization

Unix/Linux

Infrastructure as Code (IaC)

Healthcare data domain knowledge

Tools

Terraform

GitHub Actions

Job description

Overview

We are looking for a Senior Data Engineer to design, build, and optimize large-scale data processing systems supporting healthcare analytics and operational reporting. This role will involve working closely with DataOps, DevOps, and QA teams to enable scalable and reliable data pipelines.

Responsibilities

• Design and implement ETL/ELT pipelines using Python and PySpark
• Develop scalable data workflows using Apache Spark and AWS Glue
• Collaborate with QA and DevOps to integrate CI/CD and testing automation
• Manage data lake structures and ensure data quality, lineage, and auditability
• Optimize and monitor performance of batch and streaming pipelines
• Build infrastructure as code (IaC) using tools like Terraform, GitHub Actions
• Work across structured, semi-structured, and unstructured healthcare datasets

Required Technical Skills

• Core & Deep Knowledge Assessment:
  • Python
  • PySpark
  • SQL (including Window functions and CASE)
  • AWS Glue, S3, Lambda
  • Apache Spark
  • Apache Airflow
  • Delta Lake/Data Lakehouse Architecture
  • CI/CD (Terraform, GitHub Actions)
  • ETL/ELT pipeline design and optimization
• Basic Overall Knowledge Assessment:
  • Kafka
  • Data modeling and normalization
  • Unix/Linux
  • Infrastructure as Code (IaC)
  • Cloud storage, IAM, and networking fundamentals (AWS)
  • Git version control
  • Healthcare data domain knowledge