Data Engineer-II

Questhiring; Haryana; Hybrid; INR 1,800,000 - 3,000,000; Full time

Benefits offered by this job

Data-focused role

Job summary

Questhiring is seeking a hands-on Data Engineer II with strong GCP experience to design, build, and operate scalable data pipelines.

You will own end-to-end solutions from design to production support, collaborating with data, platform, and DevOps teams to deliver reliable data platforms.

The role focuses on BigQuery, Airflow, and modern data transformations, with emphasis on performance, cost, and data quality.

Qualifications

• 4+ years of hands-on Data Engineering experience.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL – complex joins, CTEs, window functions, and query optimization.
• Strong Python development skills.
• Experience with production ETL/ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience with Dataflow / Apache Beam or Dataproc / Spark.
• Strong understanding of data modelling and large-scale datasets.
• Experience with incremental and idempotent processing.
• Understanding of data contracts, schema evolution, data quality, and reconciliation.
• Experience with Git, automated testing, and CI/CD.
• Strong production troubleshooting and problem-solving skills.

Responsibilities

• Design, develop, and maintain production-grade data pipelines on GCP.
• Build scalable batch and incremental processing solutions.
• Develop transformations using BigQuery, SQL, and Dataform/dbt.
• Build workflows with Cloud Composer / Apache Airflow.
• Develop reusable transformation components and orchestration patterns.
• Build distributed data-processing with Dataflow / Beam and Dataproc / Spark.
• Design data models for analytics and downstream products.
• Implement data-quality checks, validation, reconciliation, and lineage.
• Optimize pipelines and BigQuery workloads for performance and cost.
• Implement automated testing and CI/CD integration for data workloads.
• Monitor production pipelines and troubleshoot via root-cause analysis.
• Collaborate with Data Engineering, Platform Engineering, DevOps, Architecture and business teams.

Skills

GCP

BigQuery

Advanced SQL

Python

ETL/ELT pipelines

Cloud Composer / Airflow

dbt / Dataform

Dataflow / Spark

Data Modelling

CI/CD

Git

Troubleshooting

Tools

Cloud Composer / Apache Airflow

Dataform

dbt

Dataflow / Apache Beam

Dataproc / Spark

Git

Job description

Data Engineer II – GCP Data Platform

Location: Gurgaon

Work Mode: Hybrid

Experience: 4+ Years

About the Role

We are looking for a hands-on Data Engineer II with strong GCP experience to build scalable data pipelines, transformations, and reusable data-platform capabilities.

The role involves taking solutions end-to-end—from design and development to testing, deployment, and production support—while building reliable and reusable data engineering solutions.

Key Responsibilities

• Design, develop, and maintain production-grade data pipelines on GCP.
• Build scalable batch and incremental processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, frameworks, and orchestration patterns.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data products.
• Implement data-quality checks, validation, reconciliation, and data lineage.
• Optimize pipelines and BigQuery workloads for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD.
• Monitor production pipelines and troubleshoot issues through root-cause analysis.
• Collaborate with Data Engineering, Platform Engineering, DevOps, Architecture, and business teams.

Must-Have Skills

• 4+ years of hands-on Data Engineering experience.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL – complex joins, CTEs, window functions, and query optimization.
• Strong Python development skills.
• Experience with production ETL/ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling and large-scale datasets.
• Experience with incremental and idempotent processing.
• Understanding of data contracts, schema evolution, data quality, and reconciliation.
• Experience with Git, automated testing, and CI/CD.
• Strong production troubleshooting and problem-solving skills.

Technical Skills

Cloud: GCP, BigQuery, Google Cloud Storage

Programming: Python, SQL

Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark

Orchestration: Cloud Composer, Apache Airflow

Transformation: Dataform, dbt

Engineering: Git, CI/CD, Automated Testing, Monitoring

Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata & Lineage

Good to Have

• Apache Iceberg or modern lakehouse technologies.
• Streaming or event-driven processing.
• Data Product / Data Mesh concepts.
• Metadata-driven or configuration-driven processing.
• Change Data Capture (CDC).
• Terraform / Infrastructure as Code.
• Experience building reusable components for multiple engineering teams.

What We’re Looking For

A strong ownership mindset, good analytical and debugging skills, ability to independently design solutions, and the confidence to take data engineering solutions through production.