Data Engineer II - GCP Data Platform

Deutsche Telekom Digital Labs; Gurugram District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Deutsche Telekom Digital Labs in Gurugram is seeking a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations, and reusable data-platform capabilities.

You will work across data engineering and data platform engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases.

Qualifications

• 4+ years of hands-on data engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning.
• Strong Python development skills for data processing, automation, and testing.
• Experience developing production ETL / ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization, and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing, and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Responsibilities

• Design, develop, and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries, and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation, and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimize BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with data engineers, platform engineers, DevOps, architects, and business stakeholders.

Skills

BigQuery

SQL

Python

Dataflow / Apache Beam

dbt / Dataform

Airflow / Cloud Composer

PySpark

Data modeling

CI/CD

GCP

Tools

BigQuery

Cloud Composer

dbt

Dataform

Apache Airflow

Dataflow

Dataproc

Job description

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations, and reusable data-platform capabilities. You will work across data engineering and data platform engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases. The role requires strong hands-on development skills and the ability to take a solution from design through implementation, testing, deployment, and production support.

Responsibilities

• Design, develop, and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries, and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation, and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimize BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with data engineers, platform engineers, DevOps, architects, and business stakeholders.

Requirements

• 4+ years of hands-on data engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning.
• Strong Python development skills for data processing, automation, and testing.
• Experience developing production ETL / ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization, and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing, and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Technical Skills

• Cloud and Storage: Google Cloud Platform, BigQuery, Google Cloud Storage.
• Transformation and Orchestration: Advanced SQL, Dataform / dbt, Cloud Composer / Apache Airflow, Data Processing, Dataflow / Apache Beam, Dataproc / Spark / PySpark.
• Programming and Engineering: Python, Git, automated testing, CI/CD, monitoring, and troubleshooting.
• Data Engineering Capabilities: ETL/ELT, batch processing, incremental and idempotent pipelines, data modelling, data contracts, data quality and reconciliation, metadata and lineage, query performance optimization, cloud-cost optimization, and reusable data-platform components.

Good to Have

• Experience with Apache Iceberg or modern lakehouse table formats.
• Experience with streaming or event-driven processing.
• Practical understanding of Data Product / Data Mesh principles.
• Experience with metadata-driven or configuration-driven processing.
• Experience with Change Data Capture.
• Familiarity with Terraform / Infrastructure as Code.
• Experience building reusable components used by multiple engineering teams.
• Strong Interpersonal Skills.
• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging, and problem-solving skills.
• Ability to communicate technical decisions clearly.Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.