Lead GCP Data Engineer

Kanini Software Solutions; Pune District, Chennai District, Bengaluru; On-site; INR 2,800,000 - 5,500,000; Full time

Benefits offered by this job

Award-winning team

Growth opportunities

Cutting-edge tools

Learning & development

Job summary

Kanini Software Solutions is seeking an experienced Lead Data Engineer to design, build, and maintain scalable data processing systems on the Google Cloud Platform. The role emphasizes hands-on work with BigQuery, Dataflow, Airflow and advanced SQL, plus Python or Java coding.

You will collaborate with analytics, data science and product teams to ensure data quality, governance and efficient pipelines across real-time and batch workloads. The position is hybrid with multiple Indian locations.

Qualifications

• Strong hands-on experience with Google Cloud Platform (GCP) services and data engineering best practices.
• Proficient in BigQuery, Dataflow, Airflow and SQL for scalable data pipelines.
• Programming experience in Python; basic Java knowledge is beneficial.

Responsibilities

• Design and develop scalable data pipelines on Google Cloud Platform.
• Build and optimize datasets using BigQuery and advanced SQL.
• Implement real-time and batch data processing with Dataflow.
• Orchestrate workflows using Airflow/Cloud Composer.
• Write production-grade code in Python or Java.
• Optimize query performance and cost of data pipelines.
• Collaborate with analytics, data science and product teams.
• Ensure data quality, reliability and governance in production.

Skills

GCP ecosystem

BigQuery

Dataflow

Airflow

Python

Java

Tools

Cloud Composer

Dataproc

dbt

Dataform

Job description

Role: Lead Data Engineer

Experience: 8+ Years

Location: Chennai | Pune | Coimbatore | Bangalore (Hybrid)

Job Summary

We are looking for an experienced Lead Data Engineer with deep expertise in the Google Cloud Platform ecosystem to design, build, and maintain scalable data processing systems. The ideal candidate should have strong hands‑on experience with BigQuery, Dataflow, Airflow, and advanced SQL, alongside programming capabilities in Python or Java.

Key Responsibilities

• Design and develop scalable data pipelines on Google Cloud Platform.
• Build and optimize datasets using BigQuery and advanced SQL.
• Implement real‑time and batch data processing using Dataflow.
• Implement workflow orchestration using Airflow / Cloud Composer.
• Write efficient, production‑ready code using Python or Java.
• Optimize BigQuery queries and pipeline execution paths for performance and cost.
• Work closely with analytics, data science, and product teams.
• Ensure data quality, reliability, and governance standards.
• Troubleshoot and resolve pipeline failures in production.

Mandatory Skills

• Strong experience with the Google Cloud Platform (GCP) ecosystem.
• Hands‑on expertise in BigQuery and advanced SQL.
• Experience with Dataflow for stream and batch data processing.
• Experience with Airflow (preferably GCP Cloud Composer).
• Strong coding skills in Python.
• Basic Knowledge in Java

Good to Have

• Experience with Dataform or dbt (data transformation tools).
• Experience with Dataproc or other GCP data services.
• Knowledge of GCP security, IAM, and data governance tools.
• Experience handling large-scale data and complex transformations.

Why Join Us?

• Join KANINIs award‑winning Data Engineering Team, recognized as the "Outstanding Data Engineering Team" at DES 2025.
• Work on real‑world problems at the intersection of data, AI, and product development.
• Be part of a collaborative, innovative, and growth‑driven environment.
• Access to cutting‑edge tools, leadership support, and continuous learning opportunities.