Data Engineer (Platform Focus)

Comviva; Gurugram District; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Comviva is seeking a Data Engineer to build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data, and to develop structured datasets for analytics, reporting, and downstream applications.

You will write and optimize SQL for large-scale datasets, implement data transformations with Python or Groovy, and work with Spark, Airflow, NiFi to build scalable workflows while collaborating with stakeholders.

Qualifications

• 2–4 years of experience in Data Engineering, Backend Engineering, or a related role.
• Strong proficiency in Python, Groovy, or similar scripting languages.
• Good understanding of ETL/ELT pipeline development and data transformation techniques.
• Hands‑on experience with SQL and relational databases.
• Experience with OLAP databases such as ClickHouse, Amazon Redshift, or Snowflake is preferred.
• Experience working with Apache Spark, Apache Airflow, or similar distributed data processing and orchestration frameworks.
• Familiarity with data ingestion tools such as Apache NiFi or equivalent.
• Experience working with Amazon S3, MinIO, or other object storage systems.
• Good understanding of data modeling principles for analytics and reporting.
• Strong analytical and problem-solving skills.
• Good communication and collaboration skills.

Responsibilities

• Build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data.
• Develop and maintain structured datasets for analytics, reporting, and downstream applications.
• Write and optimize SQL queries to process large-scale datasets efficiently.
• Implement data transformation logic using Python or Groovy.
• Work with Spark, Airflow, NiFi, and other platform components to build scalable data workflows.
• Integrate data from multiple internal and external sources.
• Perform data validation and identify data quality issues.
• Troubleshoot pipeline failures and optimize performance.
• Participate in designing scalable data models and warehouse structures.
• Collaborate with senior engineers, product teams, and business stakeholders to deliver data solutions.
• Contribute to automation, monitoring, and operational excellence of the data platform.
• Follow engineering best practices for testing, documentation, and code quality.

Skills

Python programming

Groovy scripting

SQL proficiency

Data modeling

Data ingestion

ETL pipelines

Spark

Airflow

NiFi

S3/MinIO

Tools

Apache Spark

Apache Airflow

Apache NiFi

ClickHouse

Amazon Redshift

Snowflake

Amazon S3/MinIO

Job description

• Build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data
• Develop and maintain structured datasets for analytics, reporting, and downstream applications
• Write and optimize SQL queries to process large-scale datasets efficiently
• Implement data transformation logic using Python or Groovy
• Work with Spark, Airflow, NiFi, and other platform components to build scalable data workflows
• Integrate data from multiple internal and external sources
• Perform data validation and identify data quality issues
• Troubleshoot pipeline failures and optimize performance
• Participate in designing scalable data models and warehouse structures
• Collaborate with senior engineers, product teams, and business stakeholders to deliver data solutions
• Contribute to automation, monitoring, and operational excellence of the data platform
• Follow engineering best practices for testing, documentation, and code quality

Key Responsibilities

• Build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data
• Develop and maintain structured datasets for analytics, reporting, and downstream applications
• Write and optimize SQL queries to process large-scale datasets efficiently
• Implement data transformation logic using Python or Groovy
• Work with Spark, Airflow, NiFi, and other platform components to build scalable data workflows
• Integrate data from multiple internal and external sources
• Perform data validation and identify data quality issues
• Troubleshoot pipeline failures and optimize performanceParticipate in designing scalable data models and warehouse structures
• Collaborate with senior engineers, product teams, and business stakeholders to deliver data solutions
• Contribute to automation, monitoring, and operational excellence of the data platform
• Follow engineering best practices for testing, documentation, and code quality

Key Requirements

• 2–4 years of experience in Data Engineering, Backend Engineering, or a related role
• Strong proficiency in Python, Groovy, or similar scripting languages
• Good understanding of ETL/ELT pipeline development and data transformation techniques
• Hands‑on experience with SQL and relational databases
• Experience with OLAP databases such as ClickHouse, Amazon Redshift, or Snowflake is preferred
• Experience working with Apache Spark, Apache Airflow, or similar distributed data processing and orchestration frameworks
• Familiarity with data ingestion tools such as Apache NiFi or equivalent
• Experience working with Amazon S3, MinIO, or other object storage systems
• Good understanding of data modeling principles for analytics and reporting
• Strong analytical and problem-solving skills
• Good communication and collaboration skills