GCP Data Engineer

Teknuova; Gurugram District, Bengaluru, Mumbai; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Teknuova is seeking a Data Engineer to design, build, and maintain scalable data pipelines on Google Cloud Platform. You will work with cross-functional teams to ensure data reliability and accessibility for downstream use cases.

Responsibilities include developing ETL/ELT pipelines with Dataflow, Dataproc, Cloud Composer, and Pub/Sub, using BigQuery for data modeling and optimization, and maintaining data storage in Cloud Storage. Strong Python and SQL skills are essential.

Qualifications

• Strong hands-on experience with Google Cloud Platform.
• Experience with BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
• Strong programming skills in Python.
• Strong proficiency in SQL.
• Experience designing and building ETL/ELT pipelines.
• Knowledge of data warehousing, data modeling, and distributed processing.
• Experience with large-scale structured and unstructured datasets.
• Knowledge of Apache Beam or Spark frameworks.
• Familiarity with Git/version control.
• Strong problem-solving and debugging skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform (GCP).
• Build and manage batch and real-time data processing workflows.
• Develop data ingestion, transformation, and integration across sources.
• Work with BigQuery to design data models and optimize queries.
• Build ETL/ELT pipelines using Dataflow, Dataproc, Cloud Composer, and Pub/Sub.
• Manage and process data stored in Google Cloud Storage.
• Ensure data quality, security, and reliability across platforms.
• Monitor and troubleshoot data pipelines and production workflows.
• Optimize cloud infrastructure and data processing workloads for performance and cost.
• Collaborate with data scientists, analysts, software engineers, and stakeholders.
• Implement CI/CD and automation for data workflows.
• Maintain detailed documentation for pipelines and architecture.

Skills

Google Cloud Platform

BigQuery

Cloud Storage

Dataflow

Dataproc

Pub/Sub

Cloud Composer

Python

SQL

ETL/ELT Pipelines

Data warehousing

Data modeling

Apache Beam

Spark

Git

Tools

Apache Airflow

Terraform

Docker

Kubernetes

dbt

Kafka

Job description

We are looking for a Data Engineer to design, build, and maintain scalable data pipelines and cloud-based data platforms on Google Cloud Platform. You will work with engineering, analytics, and business teams to ensure data is reliable, accessible, and optimized for downstream use cases.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform (GCP)
• Build and manage batch and real-time data processing workflows
• Develop data ingestion, transformation, and integration solutions across multiple data sources
• Work with BigQuery to design data models, optimize queries, and improve data warehouse performance
• Build ETL/ELT pipelines using services such as Dataflow, Dataproc, Cloud Composer, and Pub/Sub
• Manage and process data stored in Google Cloud Storage
• Ensure data quality, consistency, security, and reliability across data platforms
• Monitor and troubleshoot data pipelines and production workflows
• Optimize cloud infrastructure and data processing workloads for performance and cost
• Collaborate with data scientists, analysts, software engineers, and business stakeholders
• Implement CI/CD and automation practices for data engineering workflows
• Maintain technical documentation for data pipelines, architecture, and processes

Required Skills

• Strong hands-on experience with Google Cloud Platform
• Experience with BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer
• Strong programming skills in Python
• Strong proficiency in SQL
• Experience designing and building ETL/ELT pipelines
• Good understanding of data warehousing, data modeling, and distributed data processing
• Experience working with large-scale structured and unstructured datasets
• Knowledge of Apache Beam, Spark, or similar data processing frameworks
• Familiarity with Git and version control
• Strong problem-solving and debugging skills

Good to Have

• Experience with Apache Airflow
• Knowledge of Terraform or Infrastructure as Code
• Experience with Docker and Kubernetes
• Familiarity with Kafka or other streaming technologies
• Exposure to dbt
• Experience implementing data governance and data security practices
• Knowledge of GCP IAM, networking, and cloud security
• Google Cloud certification such as Professional Data Engineer