Gcp Data Engineer

Elabs Infotech; Hyderabad; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Elabs Infotech is seeking a GCP Data Engineer to design and implement scalable data pipelines on Google Cloud Platform. You will work with Dataflow, Dataproc, BigQuery, Cloud Storage and Airflow to process data efficiently.

The role requires strong Python, PySpark and SQL skills to develop and optimize ETL/ELT workflows, perform data validation, and automate tasks with Shell scripting. You will monitor pipelines and optimize cloud data workloads in a fast-paced environment.

Qualifications

• Experience building scalable data pipelines on Google Cloud Platform.
• Strong understanding of ETL/ELT design and data modeling on cloud platforms.
• Proficient in Python, PySpark and SQL for data processing.

Responsibilities

• Design and develop data pipelines using GCP Dataflow, Dataproc, BigQuery, Cloud Storage and Airflow.
• Develop data processing solutions using Python, PySpark and SQL.
• Build and optimize ETL/ELT workflows and data pipelines.
• Perform data integration, transformation and validation.
• Use Shell scripting for automation and operational tasks.
• Monitor, troubleshoot and optimize data pipelines and cloud data workloads.

Skills

GCP Dataflow

Dataproc

BigQuery

Cloud Storage

Airflow

Python

PySpark

SQL

ETL/ELT design

Shell scripting

Job description

We are looking for a GCP Data Engineer with strong experience in building scalable data pipelines and ETL/ELT solutions using Google Cloud Platform.

Key Responsibilities:

• Design and develop data pipelines using GCP Dataflow, Dataproc, BigQuery, Cloud Storage and Airflow.
• Develop data processing solutions using Python, PySpark and SQL.
• Build and optimize ETL/ELT workflows and data pipelines.
• Perform data integration, transformation and validation.
• Use Shell scripting for automation and operational tasks.
• Monitor, troubleshoot and optimize data pipelines and cloud data workloads.