Data Engineer

Cohere Health; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Verana Health is seeking a Data Engineer to build and optimize scalable data pipelines on an AWS-based lakehouse platform. You will work with tools like Apache Iceberg, AWS Glue, and Athena while collaborating with senior engineers and analytics teams to deliver efficient data solutions. The ideal candidate should have 4–5 years of experience, strong skills in Python, SQL, and AWS technologies, and a thorough understanding of data validation and optimization. This role is based in Hyderabad, India.

Qualifications

• 4–5 years of experience in data engineering.
• Strong hands-on experience with Python, SQL, PySpark/Spark.
• Experience with AWS S3, Glue, Athena.
• Experience with DBT and Airflow (or equivalent).
• Familiarity with Apache Iceberg / Hudi / Delta.

Responsibilities

• Develop and maintain batch data pipelines.
• Build reusable ingestion and transformation workflows.
• Build and maintain Apache Iceberg datasets.
• Support schema evolution and partition optimization.
• Optimize Athena queries and troubleshoot performance bottlenecks.
• Implement data validation checks and monitoring.

Skills

Python

SQL

PySpark/Spark

AWS S3

Glue

Athena

DBT

Airflow

Apache Iceberg

Job description

Opportunity Overview

We are looking for a Data Engineer to build and optimize scalable data pipelines and analytical datasets on our AWS-based lakehouse platform.

• You will work with Apache Iceberg, AWS Glue, Athena, dbt, and Airflow to support reliable and efficient data processing for analytics and operational use cases.
• You will collaborate with senior engineers, analytics teams, and platform stakeholders to deliver production-grade data solutions.

What you’ll do

• Develop and maintain batch data pipelines
• Build reusable ingestion and transformation workflows
• Build and maintain Apache Iceberg datasets
• Support schema evolution and partition optimization
• Optimize Athena queries and troubleshoot performance bottlenecks
• Implement data validation checks and monitoring
• Support production issue troubleshooting

Required Qualifications

• 4–5 years of experience in data engineering
• Strong hands‑on experience with Python, SQL, PySpark/Spark
• Experience with AWS S3, Glue, Athena
• Experience with DBT and Airflow (or equivalent)
• Familiarity with Apache Iceberg / Hudi / Delta
• Understanding of partitioning and Parquet optimization

Equal Opportunity Statement

Cohere Health is an Equal Opportunity Employer. We are committed to fostering an environment of mutual respect where equal employment opportunities are available to all. To us, it’s personal.