Data Engineer

Questhiring; Haryana; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Questhiring in India is hiring a hands-on Data Engineer II to join our GCP Data Platform. You will build scalable pipelines, transformations and reusable components across teams.

You will work across Data Engineering and Platform Engineering, delivering production-grade data products, taking solutions from design through testing, deployment and production support. The role requires strong Python, SQL, BigQuery and Airflow experience, with a focus on data quality, metadata, contracts and cost

Qualifications

• 4+ years hands-on data engineering experience building production data solutions.
• Strong GCP and BigQuery expertise.
• Advanced SQL skills including complex joins, CTEs and window functions.
• Strong Python development skills for data processing and testing.
• Experience developing production ETL/ELT pipelines.
• Hands-on experience with Cloud Composer / Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark.
• Strong understanding of data modelling, including normalization and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with data quality validation and reconciliation.
• Experience metadata and data lineage.

Responsibilities

• Design, implement and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop reusable transformation components, orchestration patterns, libraries and templates.
• Implement metadata-driven processing and data contracts where appropriate.
• Build and optimize data-processing workflows using Dataflow/Airflow/Dataproc.
• Collaborate with data engineers, platform engineers, DevOps and stakeholders.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform components and standards.

Skills

GCP Experience

BigQuery

SQL

Python

ETL/ELT

Airflow

Dataform/dbt

Dataflow/Beam

Data Modelling

CI/CD

Git

Tools

Git

Cloud Composer

CI/CD tools

Job description

Role: Data Engineer II - GCP Data Platform

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations and reusable data-platform capabilities.

Objective of the Role

You will work across Data Engineering and Data Platform Engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases.

The role requires strong hands-on development skills and the ability to take a solution from design through implementation, testing, deployment and production support.

You Will

• Design, develop and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL and Dataform / dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimise BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with Data Engineers, Platform Engineers, DevOps, Architects and business stakeholders.

You Must Have

• 4+ years of hands-on Data Engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills including complex joins, CTEs, window functions and query tuning.
• Strong Python development skills for data processing, automation and testing.
• Experience developing production ETL / ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Technical Skills

Cloud & Storage

• Google Cloud Platform
• BigQuery
• Google Cloud Storage

Transformation & Orchestration

• Advanced SQL
• Dataform / dbt
• Cloud Composer / Apache Airflow

Data Processing

• Dataflow / Apache Beam
• Dataproc / Spark / PySpark

Programming & Engineering

• Python
• Git
• Automated testing
• CI/CD
• Monitoring and troubleshooting

Data Engineering Capabilities

• ETL / ELT
• Batch processing
• Incremental and idempotent pipelines
• Data modelling
• Data contracts
• Data quality and reconciliation
• Metadata and lineage
• Query performance optimisation
• Cloud-cost optimisation
• Reusable data-platform components

Good to Have

• Experience with Apache Iceberg or modern lakehouse table formats.
• Experience with streaming or event-driven processing.
• Practical understanding of Data Product / Data Mesh principles.
• Experience with metadata-driven or configuration-driven processing.
• Experience with Change Data Capture.
• Familiarity with Terraform / Infrastructure as Code.
• Experience building reusable components used by multiple engineering teams.

Strong Interpersonal Skills

• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging and problem-solving skills.
• Ability to communicate technical decisions clearly.
• Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.