GCP Data Engineer

Berribot; Bengaluru; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Berribot is seeking an experienced GCP Data Engineer to join our expanding team in Pan India. You will own design and delivery of scalable data pipelines and data products for BFSI clients, focusing on cost-efficient processing and analytics readiness.

You will implement ETL/ELT workflows, migrate legacy workloads to GCP, and collaborate with client teams on PoCs and production cutovers. Strong BigQuery/Dataproc/Dataflow expertise and Data IAM governance are required.

Qualifications

• Google Cloud Platform expertise (BigQuery, Dataproc, Dataflow).
• Experience with Pub/Sub, Cloud Functions, Data Fusion, Data Transfer.
• Cloud Composer (Airflow) orchestration experience.
• Data warehouse design and performance tuning.
• Cloud IAM for data security and access governance.

Responsibilities

• Design and develop scalable batch and streaming pipelines on GCP using Dataflow, Dataproc, Pub/Sub, and Cloud Functions.
• Build and optimise cloud data warehouse solutions on BigQuery, including partitioning and clustering.
• Migrate legacy workloads to GCP using Dataproc, BigQuery, and Data Fusion.
• Develop ELT/ETL workflows with Python/Java/Scala/PySpark; orchestrate with Airflow.
• Implement data security and governance and reusable validation frameworks.
• Tune Spark and BigQuery workloads for performance and cost.
• Set up CI/CD and automation for GCP data assets with Git-based pipelines.
• Collaborate with architects and client teams on PoCs and production cutover.

Skills

BigQuery

Dataproc

Dataflow

PubSub

Cloud Functions

Airflow/Composer

Data warehouse design

Performance tuning

Cloud IAM

Vertex AI

Tools

Terraform

Looker/BI tools

Vertex AI

Job description

We are looking for experienced GCP Data Engineers to join our growing team. The ideal candidate should have strong hands‑on experience in BigQuery-based data warehousing, Dataproc/Spark processing, and orchestrated, cost‑efficient pipelines to deliver scalable analytics and AI‑ready data products for BFSI clients.

Location

Pan India

Experience

6-10 Years (role-dependent), including 3+ years hands‑on GCP

Employment Type

Full-time

Key Responsibilities

• Strong experience and understanding of Data Warehouse and Data Lake platform architecture and design.
• Design and develop scalable batch and streaming pipelines on GCP using Dataflow, Dataproc (Spark), Pub/Sub, and Cloud Functions.
• Build and optimise cloud data warehouse solutions on BigQuery, including partitioning, clustering, and query cost optimisation.
• Migrate and modernise legacy workloads (Hadoop, Teradata, on‑prem ETL) to GCP using Dataproc, BigQuery, and Data Fusion.
• Develop ELT/ETL workflows with Python, Java, Scala, and PySpark; orchestrate pipelines using Cloud Composer (Airflow).
• Implement data security and governance via Cloud IAM, and build reusable data validation and reconciliation frameworks.
• Tune Spark and BigQuery workloads for performance and cost, achieving measurable runtime and storage improvements.
• Set up CI/CD and automation for GCP data assets using Git‑based pipelines and infrastructure‑as‑code.
• Collaborate with architects and client teams on solutioning, PoCs, demos, and production cutover.

Mandatory Skills

• Google Cloud Platform (BigQuery, Dataproc, Dataflow)
• Pub/Sub, Cloud Functions, Data Fusion, Data Transfer
• Cloud Composer (Airflow) orchestration
• Data warehouse & pipeline design + performance tuning
• Cloud IAM - data security & access governance

Good to Have

• CI/CD, Git, infrastructure‑as‑code (Terraform)
• BI & analytics - Looker, Power BI, Tableau
• Exposure to Vertex AI/GenAI on GCP

Your Next Career Opportunity Awaits!