Engineering Manager

Questhiring; Gurugram District; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Questhiring is seeking an experienced Data Engineering Manager to lead data engineering initiatives and deliver scalable data platforms and pipelines on GCP. You will mentor a team of data engineers, design robust data architectures, and collaborate with Product, Analytics, and Data Science to optimize performance, reliability, and cost while ensuring production-grade quality.

The ideal candidate will have strong leadership skills, deep expertise in PySpark and Airflow, and a track record of

Qualifications

• 12–16 years IT experience, with strong data engineering background.
• GCP – Mandatory.
• Hands-on PySpark / Apache Spark experience.
• Hands-on Apache Airflow experience.
• Knowledge of ETL/ELT pipelines and distributed data processing.
• Experience designing scalable cloud data platforms.
• Strong Python programming skills.
• Experience with data warehouses, data lakes, and modern cloud data architectures.
• Knowledge of data modelling, data integration, and pipeline orchestration.
• Proven leadership of technical teams and delivering enterprise-scale data projects.
• Strong communication and stakeholder management skills.

Responsibilities

• Lead and mentor a team of data engineers and provide strong technical direction.
• Design and architect scalable, secure, and high-performance data solutions on GCP.
• Drive development of robust data pipelines using PySpark and Apache Airflow.
• Own end-to-end data engineering projects from requirement analysis to production deployment.
• Define data architecture, standards, coding practices, and best practices.
• Collaborate with Product, Analytics, Data Science, and other technology teams.
• Optimize data pipelines for performance, scalability, reliability, and cost.
• Establish monitoring, logging, data quality, and operational processes for production pipelines.
• Review designs and code while ensuring engineering quality.
• Troubleshoot complex data pipeline and platform issues.
• Manage timelines, dependencies, risks, and delivery expectations.
• Stay updated with emerging cloud data engineering technologies.

Skills

GCP

PySpark

Apache Spark

Apache Airflow

Python

Data engineering concepts

ETL/ELT pipelines

Distributed data processing

Data modelling

Data integration

Pipeline orchestration

Leadership

Communication

Tools

PySpark

Apache Spark

Apache Airflow

BigQuery

Cloud Storage

Dataflow

Dataproc

Pub/Sub

Kubernetes

Docker

Terraform

Job description

About the Role

We are looking for an experienced Data Engineering Manager to lead data engineering initiatives and drive the design, development, and delivery of scalable data platforms and pipelines.

Key Responsibilities

• Lead and mentor a team of data engineers and provide strong technical direction.
• Design and architect scalable, secure, and high-performance data engineering solutions on GCP.
• Drive development of robust data pipelines using PySpark and Apache Airflow.
• Own end-to-end data engineering projects, from requirement analysis to production deployment.
• Define data architecture, engineering standards, coding practices, and best practices.
• Work closely with Product, Analytics, Data Science, and other technology teams.
• Optimize data pipelines for performance, scalability, reliability, and cost.
• Establish monitoring, logging, data quality, and operational processes for production pipelines.
• Review technical designs and code while ensuring engineering quality.
• Troubleshoot complex data pipeline and platform issues.
• Manage project timelines, technical dependencies, risks, and delivery expectations.
• Stay updated with emerging technologies and trends in cloud data engineering.

Mandatory Skills

• 12–16 years of overall IT experience, with strong experience specifically in Data Engineering.
• GCP – Mandatory
• Strong hands-on experience with PySpark / Apache Spark.
• Strong hands‑on experience with Apache Airflow.
• Strong understanding of data engineering concepts, ETL/ELT pipelines, and distributed data processing.
• Experience designing and implementing scalable data platforms on cloud.
• Strong programming/scripting experience, preferably Python.
• Experience with data warehouses, data lakes, and modern cloud data architectures.
• Strong understanding of data modelling, data integration, and pipeline orchestration.
• Proven experience leading technical teams and delivering enterprise‑scale data engineering projects.
• Strong communication, stakeholder management, and problem‑solving skills.

Good to Have

• Experience with BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub or other GCP data services.
• Experience with CI/CD and DevOps practices for data engineering.
• Knowledge of Kubernetes, Docker, Terraform, or Infrastructure as Code.
• Experience implementing data quality and governance frameworks.