GCP Data engineer with SQL, DataProc, dataflow expertise

Tredence Inc.; Bengaluru; Hybrid; INR 2,500,000 - 3,800,000; Full time

Job summary

Tredence Inc. is seeking a high-performing GCP Data Engineering Lead / Associate Manager with 4 to 9 years of experience to join our rapidly growing Data Engineering practice across our major India delivery hubs.

You will remain hands-on in architecting data platforms on GCP with PySpark, Python, and BigQuery, while guiding client delivery, mentoring engineers, and driving agile practices across delivery teams.

Qualifications

• 4–9 years in Data Engineering with delivery leadership
• Experience as Technical Lead, Module Lead, or Scrum Master for data delivery
• GCP prod-level: BigQuery, GCS, Pub/Sub, Dataproc, Cloud Functions
• Proficient in Python and PySpark / Spark SQL
• Hands-on Airflow / Cloud Composer and DAG development
• Strong SQL with query optimization and large-scale data modeling
• CI/CD, containerization (Docker/Kubernetes), and IaC (Terraform)
• Preferred: Dataflow/Beam and Google Cloud Data Engineer certification

Responsibilities

• Design, build, and optimize scalable ETL/ELT pipelines using Python, PySpark, and GCP Dataproc
• Oversee scheduling, monitoring, debugging of Airflow/Cloud Composer workflows
• Design cost-efficient data models, schemas, and partitioning in BigQuery
• Collaborate with client product owners and delivery managers to translate business problems into roadmaps
• Enforce production-grade standards including CI/CD, Docker/Kubernetes, and Terraform

Skills

Data engineering leadership

GCP data engineering

Python

PySpark / Spark SQL

Airflow / Cloud Composer

SQL optimization

CI/CD practices

Terraform

Docker / Kubernetes

Tools

BigQuery

GCS

Pub/Sub

Dataproc

Cloud Functions

Apache Airflow

Terraform

Docker

Kubernetes

Job description

We are seeking a high-performing GCP Data Engineering Lead / Associate Manager with 4 to 9 years of overall experience to join our rapidly growing Data Engineering practice across our major India delivery hubs.

Job Title: GCP Data Engineering
Experience Level: 4 to 9 Years
Locations: Bangalore, Chennai, Pune, Gurugram, Kolkata, Hyderabad (Hybrid / Office-based as per company policy)
Employment Type: Full-time

Role Overview

This is a dynamic player-coach role designed for a technical powerhouse who wants to accelerate their career into engineering management. You will remain hands-on in architecting data platforms using GCP, SQL, Python, Airflow, and PySpark, while managing client delivery, mentoring junior engineers, and driving agile engineering practices across our delivery teams.

Key Responsibilities

• Design, build, and optimize scalable enterprise ETL/ELT batch and real-time pipelines using Python, PySpark, and GCP Dataproc.
• Oversee the scheduling, monitoring, and debugging of multi-stage data workflows using Apache Airflow / Google Cloud Composer.
• Design cost-efficient data models, schemas, and partition strategies within Google BigQuery to ensure rapid analytical query execution.
• Collaborate directly with client product owners, business analysts, and Tredence delivery managers to translate complex business problems into clear technical roadmaps and sprint backlogs.
• Enforce production-grade standards including automated data quality checks, CI/CD pipelines, containerization (Docker/Kubernetes), and Infrastructure as Code (Terraform).

Required Technical Skills & Qualifications

Core Technical Stack (Must Haves):

• 4 to 9 years of dedicated experience in Data Engineering and Big Data platforms.
• Proven track record of acting as a Technical Lead, Module Lead, or Scrum Master for a data delivery track.
• Google Cloud Platform (GCP): Production-level expertise with core data services: BigQuery, Cloud Storage (GCS), Pub/Sub, Dataproc, and Cloud Functions.
• Programming & Big Data: Advanced coding skills in Python and distributed data processing utilizing PySpark / Spark SQL.
• Orchestration: Hands-on experience developing complex DAGs inside Apache Airflow / Cloud Composer.
• Advanced SQL: Deep understanding of database internals, indexing, analytical functions, and query optimization.

Preferred / Good to Have:

• Familiarity with streaming architectures using Apache Beam / Dataflow.
• Experience with Terraform for cloud infrastructure automation.
• An active Google Cloud Professional Data Engineer certification.

Soft Skills & Tredence Culture Fit

• Growth Mindset: Ability to thrive in a fast-paced, high-growth analytics consulting environment.
• Strong Communication: Articulate, empathetic communicator capable of managing both internal engineering expectations and client deliverables.
• Problem-Solving: Highly analytical mindset with a strong bias for action and removing blockers for the team.

Required Skills - GCP DE Experience, Big query, SQL, Cloud compressor/Python, Cloud functions, Dataproc+pyspark, Python injection, Dataflow+PUB/SUB

Notice period - Immediate joiners