Gcp Data Engineer

Walking Tree; Hyderabad, Pune District; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Walking Tree seeks a data pipeline developer to build scalable ETL/ELT workflows on Google Cloud Platform, leveraging BigQuery, Dataflow, Cloud Storage and Airflow.

You will migrate on-prem ETLs, optimize data models, and collaborate with cross-functional teams to deliver secure, compliant analytics solutions. Strong SQL and Python skills are required; experience with PySpark is a plus; Hyderabad-based on-site role.

Qualifications

• Extensive experience in IT data analytics projects.
• Hands-on experience migrating on-prem ETLs to GCP.
• Proficient in SQL, Python, and data modeling.
• Familiar with BigQuery, Dataflow, DataProc, and Airflow.

Responsibilities

• Design and maintain scalable ETL/ELT data pipelines on GCP.
• Migrate on-prem ETLs to Google Cloud Platform using cloud-native tools.
• Optimize BigQuery datasets for analytics and reporting.
• Collaborate with cross-functional teams to integrate data workflows.
• Automate workflows with Cloud Composer / Airflow.
• Ensure data security and governance in cloud environments.

Skills

ETL/ELT

Data pipeline design

Python

SQL

Pandas

Numpy

SciPy

Matplotlib

PySpark

BigQuery

Dataflow

Apache Airflow

Cloud Composer

GCP

Tools

BigQuery

Dataflow

Cloud Storage

Cloud Composer

Airflow

Pub/Sub

Presto SQL

Hive SQL

Python

Job description

Data Pipeline Development: Build and maintain scalable, secure ETL/ELT data workflows using BigQuery and other GCP services (e.g., Dataflow, Cloud Storage). [1, 2]Extensive Experience in IT data analytics projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Cloud Data,Proc, Google Cloud Storage, ComposerSQL concepts, Presto SQL, Hive SQL, Python (Pandas, Numpy, SciPy, Matplotlib) and PysparkDesign and implement scalable data pipelines using Google Cloud DataflowDevelop and optimize BigQuery datasets for efficient data analysis and reportingCollaborate with cross-functional teams to integrate data solutions with business processesAutomate data workflows using Cloud Composer and Apache AirflowEnsure data security and compliance with GCP Identity and Access ManagementQuery & Storage Optimization: Develop advanced SQL queries and design data models using partitioning, clustering, and materialized views to minimize costs and maximize performanceImplement real-time data processing with Google Cloud Pub/Sub and DataflowContinuously evaluate and adopt new GCP tools and technologies GCP Biq query, Python , SQL , AirflowComposer