Data Engineer

Questhiring; Gurugram District; On-site; INR 1,800,000 - 2,800,000; Full time

Job summary

Questhiring is seeking a hands-on Data Engineer II in Gurgaon, India to design, build and maintain scalable data pipelines on Google Cloud Platform. The role focuses on production-grade data products, reusable components, and robust data-platform practices.

You will work across Data Engineering and Data Platform Engineering, delivering end-to-end solutions from design to production support, with strong emphasis on performance, data quality, and CI/CD integration.

Qualifications

• 4+ years of hands-on data engineering experience building production data solutions.
• Strong experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills including complex joins, CTEs, window functions and query tuning.
• Strong Python development skills for data processing, automation and testing.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Responsibilities

• Design, develop and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL and Dataform / dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimise BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with Data Engineers, Platform Engineers, DevOps, Architects and business stakeholders.

Skills

GCP

BigQuery

SQL

Python

Dataflow

Airflow

dbt/Dataform

Spark

CI/CD

Git

Data Modeling

ETL/ELT

Data Quality

Metadata & Lineage

Testing

Tools

Cloud Composer

Apache Airflow

dbt

Dataform

Git

Job description

Objective of the Role

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations and reusable data-platform capabilities.

You will work across Data Engineering and Data Platform Engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases.

The role requires strong hands‑on development skills and the ability to take a solution from design through implementation, testing, deployment and production support.

You Will

• Design, develop and maintain production‑grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data‑processing solutions.
• Develop complex transformations using BigQuery, SQL and Dataform / dbt.
• Build and maintain data‑processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries and templates.
• Implement metadata‑driven and configuration‑driven processing where appropriate.
• Build distributed data‑processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data‑product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data‑quality checks, validation and reconciliation.
• Capture and integrate metadata and lineage into data‑processing workflows.
• Optimise BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root‑cause analysis.
• Contribute to reusable data‑platform capabilities and engineering standards.
• Collaborate with Data Engineers, Platform Engineers, DevOps, Architects and business stakeholders.

You Must Have

• 4+ years of hands‑on Data Engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands‑on experience with BigQuery.
• Advanced SQL skills including complex joins, CTEs, window functions and query tuning.
• Strong Python development skills for data processing, automation and testing.
• Hands‑on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization and dimensional modelling.
• Experience working with large‑scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data‑quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Technical Skills

• BigQuery

Transformation & Orchestration

• Advanced SQL
• Dataform / dbt
• Cloud Composer / Apache Airflow

Data Processing

• Dataflow / Apache Beam
• Python
• Git
• Automated testing
• CI/CD
• Monitoring and troubleshooting

Data Engineering Capabilities

• ETL / ELT
• Batch processing
• Incremental and idempotent pipelines
• Data quality and reconciliation
• Metadata and lineage
• Query performance optimisation

Good to Have

• Experience with Apache Iceberg or modern lakehouse table formats.
• Experience with streaming or event‑driven processing.
• Practical understanding of Data Product / Data Mesh principles.
• Experience with metadata‑driven or configuration‑driven processing.
• Experience with Change Data Capture.Familiarity with Terraform / Infrastructure as Code.
• Experience building reusable components used by multiple engineering teams.

Strong Interpersonal Skills

• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging and problem‑solving skills.
• Ability to communicate technical decisions clearly.
• Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.