Data Engineer- Lead, Chennai

Prodapt Solutions; Chennai; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

Prodapt Solutions in Chennai is seeking a skilled GCP Data Engineer to join our growing data team. You will design, build, and optimize scalable data pipelines on Google Cloud Platform, collaborating with analysts, scientists, and stakeholders across the globe.

The role emphasizes hands-on work with BigQuery, Dataproc, and Cloud Composer to deliver reliable data solutions while upholding data governance and security standards. Strong problem-solving and communication are essential.

Responsibilities

• Design, develop, and maintain robust, scalable data pipelines on Google Cloud Platform (GCP).
• Work with BigQuery for data warehousing, query optimization, and large-scale data analysis.
• Build and manage data processing workflows using Dataproc (Hadoop/Spark on GCP).
• Write, optimize, and troubleshoot complex SQL queries for data transformation, validation, and reporting.
• Design, schedule, and monitor data workflows using Cloud Composer (Apache Airflow).
• Perform root-cause analysis and troubleshooting of pipeline failures, data quality issues, and performance bottlenecks.
• Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions.
• Ensure data governance, security, and compliance standards across all GCP data engineering solutions.
• Participate in code reviews, design discussions, and knowledge-sharing sessions within the team.
• Document technical designs, data flows, and operational runbooks for supportability.

Job description

Overview:

We are looking for a skilled and detail-oriented GCP Data Engineer to join our growing data engineering team. The ideal candidate will have hands-on experience designing, building, and optimizing scalable data pipelines on Google Cloud Platform, along with strong problem-solving and communication skills to collaborate effectively with cross-functional and global stakeholders.

Key Responsibilities:

• Design, develop, and maintain robust, scalable, and efficient data pipelines on Google Cloud Platform (GCP).


• Work extensively with BigQuery (BQ) for data warehousing, query optimization, and large-scale data analysis.


• Build and manage big data processing workflows using Dataproc (Hadoop/Spark ecosystem on GCP).


• Write, optimize, and troubleshoot complex SQL queries for data transformation, validation, and reporting.


• Design, schedule, and monitor data workflows using Cloud Composer (Apache Airflow).


• Perform root-cause analysis and troubleshooting of pipeline failures, data quality issues, and performance bottlenecks in production environments.


• Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions.


• Ensure data governance, security, and compliance standards are followed across all GCP data engineering solutions.


• Participate in code reviews, design discussions, and knowledge-sharing sessions within the team.


• Document technical designs, data flows, and operational runbooks for supportability.