Senior Data Engineer

Questhiring; Gurugram District; Hybrid; INR 1,500,000 - 2,600,000; Full time

Job summary

Questhiring in Gurgaon is seeking a hands-on Senior Data Engineer II with strong GCP experience to build scalable data pipelines and reusable data-platform capabilities. You will own end-to-end delivery—from design and development to testing, deployment, and production support.

The role focuses on BigQuery, SQL, Dataform/dbt, Airflow, and end-to-end data engineering patterns, collaborating with cross-functional teams in a hybrid work environment.

Qualifications

• 6+ years of hands-on Data Engineering experience.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL – complex joins, CTEs, window functions, and query optimization.
• Strong Python development skills.
• Experience with production ETL/ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling and large-scale datasets.
• Experience with incremental and idempotent processing.
• Understanding of data contracts, schema evolution, data quality, and reconciliation.
• Experience with Git, automated testing, and CI/CD.
• Strong production troubleshooting and problem-solving skills.

Responsibilities

• Design, develop, and maintain production-grade data pipelines on GCP.
• Build scalable batch and incremental processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, frameworks, and orchestration patterns.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data products.
• Implement data-quality checks, validation, reconciliation, and data lineage.
• Optimize pipelines and BigQuery workloads for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD.
• Monitor production pipelines and troubleshoot issues through root-cause analysis.
• Collaborate with Data Engineering, Platform Engineering, DevOps, Architecture, and business teams.

Skills

Data Engineering

GCP

BigQuery

SQL

Python

ETL/ELT

Airflow

dbt

Dataform

Dataflow

Dataproc

Spark

Data Modelling

Incremental Processing

CI/CD

Data Quality

Tools

Cloud Composer

Apache Airflow

Git

Dataproc

Apache Beam

PySpark

Job description

About the job

Senior Data Engineer - GCP Data Platform

Location: Gurgaon

Work Mode: Hybrid

Experience: 6+ Years

About the Role

We are looking for a hands-on Data Engineer II with strong GCP experience to build scalable data pipelines, transformations, and reusable data-platform capabilities.

The role involves taking solutions end-to-end—from design and development to testing, deployment, and production support—while building reliable and reusable data engineering solutions.

Key Responsibilities

• Design, develop, and maintain production-grade data pipelines on GCP.
• Build scalable batch and incremental processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, frameworks, and orchestration patterns.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data products.
• Implement data-quality checks, validation, reconciliation, and data lineage.
• Optimize pipelines and BigQuery workloads for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD.
• Monitor production pipelines and troubleshoot issues through root-cause analysis.
• Collaborate with Data Engineering, Platform Engineering, DevOps, Architecture, and business teams.

Must-Have Skills

• 6+ years of hands-on Data Engineering experience.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL – complex joins, CTEs, window functions, and query optimization.
• Strong Python development skills.
• Experience with production ETL/ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling and large-scale datasets.
• Experience with incremental and idempotent processing.
• Understanding of data contracts, schema evolution, data quality, and reconciliation.
• Experience with Git, automated testing, and CI/CD.
• Strong production troubleshooting and problem-solving skills.

Technical Skills

Cloud: GCP, BigQuery, Google Cloud Storage

Programming: Python, SQL

Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark

Orchestration: Cloud Composer, Apache Airflow

Transformation: Dataform, dbt

Engineering: Git, CI/CD, Automated Testing, Monitoring

Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata & Lineage

Good to Have

• Apache Iceberg or modern lakehouse technologies.
• Streaming or event-driven processing.
• Data Product / Data Mesh concepts.
• Metadata-driven or configuration-driven processing.
• Change Data Capture (CDC).
• Terraform / Infrastructure as Code.
• Experience building reusable components for multiple engineering teams.

What We're Looking For

A strong ownership mindset, good analytical and debugging skills, ability to independently design solutions, and the confidence to take data engineering solutions through production.