GCP Data engineer with SQL, DataProc, dataflow expertise

Tredence Inc.; Pune District; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Tredence Inc. in India is seeking a GCP Data Engineering Lead/Associate Manager with 4–9 years of experience. The role combines hands-on data platform engineering with engineering leadership to deliver client-ready data solutions.

You will architect scalable pipelines, mentor junior engineers, and drive agile practices across delivery teams in multiple hubs including Pune, Bangalore, Chennai, and more. Hybrid/office-based work per policy.

Qualifications

• 4 to 9 years of experience in Data Engineering / Big Data
• Proven track record as Technical Lead or Scrum Master for data delivery
• GCP production-level experience with BigQuery, GCS, Pub/Sub, Dataproc, Cloud Functions
• Python and PySpark / Spark SQL for distributed processing
• Hands-on DAG development in Airflow / Cloud Composer
• Advanced SQL and query optimization
• Familiarity with streaming architectures and Terraform
• Active Google Cloud Professional Data Engineer certification (preferred)
• Excellent communication and client-delivery mindset

Responsibilities

• Design, build, and optimize scalable ETL/ELT pipelines using Python, PySpark and GCP Dataproc
• Oversee scheduling, monitoring, and debugging of multi-stage data workflows with Airflow/Composer
• Design cost-efficient data models, schemas, and partition strategies in BigQuery
• Translate complex business problems into technical roadmaps and sprint backlogs
• Enforce CI/CD, containerization (Docker/Kubernetes) and IaC (Terraform) across delivery teams

Skills

GCP Data Eng

PySpark

Airflow

SQL

Big Data

Terraform

Docker

Kubernetes

Data Modeling

Tools

Dataproc

BigQuery

Cloud Composer

Pub/Sub

Terraform

Cloud Functions

Job description

We are seeking a high-performing GCP Data Engineering Lead / Associate Manager with 4 to 9 years of overall experience to join our rapidly growing Data Engineering practice across our major India delivery hubs.

Job Title: GCP Data Engineering
Experience Level: 4 to 9 Years
Locations: Bangalore, Chennai, Pune, Gurugram, Kolkata, Hyderabad (Hybrid / Office-based as per company policy)
Employment Type: Full-time

Role Overview

This is a dynamic player-coach role designed for a technical powerhouse who wants to accelerate their career into engineering management. You will remain hands-on in architecting data platforms using GCP, SQL, Python, Airflow, and PySpark, while managing client delivery, mentoring junior engineers, and driving agile engineering practices across our delivery teams.

Key Responsibilities

Design, build, and optimize scalable enterprise ETL/ELT batch and real-time pipelines using Python, PySpark, and GCP Dataproc.

Oversee the scheduling, monitoring, and debugging of multi-stage data workflows using Apache Airflow / Google Cloud Composer.

Design cost-efficient data models, schemas, and partition strategies within Google BigQuery to ensure rapid analytical query execution.

Collaborate directly with client product owners, business analysts, and Tredence delivery managers to translate complex business problems into clear technical roadmaps and sprint backlogs.

Enforce production-grade standards including automated data quality checks, CI/CD pipelines, containerization (Docker/Kubernetes), and Infrastructure as Code (Terraform).

Required Technical Skills & Qualifications

Core Technical Stack (Must Haves):

4 to 9 years of dedicated experience in Data Engineering and Big Data platforms.

Proven track record of acting as a Technical Lead, Module Lead, or Scrum Master for a data delivery track.

Google Cloud Platform (GCP): Production-level expertise with core data services: BigQuery, Cloud Storage (GCS), Pub/Sub, Dataproc, and Cloud Functions.

Programming & Big Data: Advanced coding skills in Python and distributed data processing utilizing PySpark / Spark SQL.

Orchestration: Hands-on experience developing complex DAGs inside Apache Airflow / Cloud Composer.

Advanced SQL: Deep understanding of database internals, indexing, analytical functions, and query optimization.

Preferred / Good to Have:

Familiarity with streaming architectures using Apache Beam / Dataflow.

Experience with Terraform for cloud infrastructure automation.

An active Google Cloud Professional Data Engineer certification.

Soft Skills & Tredence Culture Fit

Growth Mindset: Ability to thrive in a fast-paced, high-growth analytics consulting environment.

Strong Communication: Articulate, empathetic communicator capable of managing both internal engineering expectations and client deliverables.

Problem-Solving: Highly analytical mindset with a strong bias for action and removing blockers for the team.

Required Skills

Skills

Required Skills - GCP DE Experience, Big query, SQL, Cloud compressor/Python, Cloud functions, Dataproc+pyspark, Python injection, Dataflow+PUB/SUB

Notice period - Immediate joiners