Gcp Data Engineer

Pyramid It Consulting; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Pyramid It Consulting is seeking a hands-on Data Engineer to design, build, and optimize scalable data pipelines on Google Cloud Platform using Scala. You will work with global clients, applying ETL/ELT patterns and ensuring data quality and performance.

Ideal candidates have 4+ years in data engineering, strong GCP and Scala skills, and experience with BigQuery and Dataflow. Collaboration with data scientists and stakeholders is essential for analytics and ML use cases.

Qualifications

• 4+ years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Proficiency in Scala (mandatory).
• Solid experience with BigQuery and Dataflow / Apache Beam.
• Strong understanding of distributed systems, data modeling, and performance tuning.
• Experience building ETL/ELT pipelines for large-scale datasets.
• Familiarity with SQL, NoSQL, and cloud-based data warehouses.
• Knowledge of CI/CD pipelines, version control (Git), and Agile methodologies.
• Strong problem-solving and communication skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform.
• Build and optimize batch and streaming data processing frameworks using Scala.
• Develop ETL/ELT workflows using GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Composer.
• Collaborate with data scientists, analysts, and stakeholders to support analytics and ML use cases.
• Ensure data quality, reliability, and performance across pipelines.
• Implement best practices for cloud security, governance, and cost optimization.
• Troubleshoot production issues and perform root cause analysis.
• Participate in architecture discussions, code reviews, and DevOps processes.

Skills

Data Engineering

GCP

Scala

BigQuery

Dataflow/Apache Beam

Distributed systems

ETL/ELT pipelines

SQL/NoSQL

CI/CD/Git/Agile

Problem-solving/communication

Job description

Role Overview

We are looking for a hands-on Data Engineer with strong expertise in Google Cloud Platform (GCP) and Scala to join our Data Engineering team. The ideal candidate will design, build, and optimize scalable data pipelines and platforms that enable advanced analytics and AI solutions for global clients.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform
• Build and optimize batch and streaming data processing frameworks using Scala
• Develop ETL/ELT workflows using GCP services such as:
  • BigQuery
  • Dataflow (Apache Beam)
  • Pub/Sub
  • Cloud Storage
  • Cloud Composer (Airflow)
• Collaborate with data scientists, analysts, and business stakeholders to support analytics and ML use cases
• Ensure data quality, reliability, and performance across pipelines
• Implement best practices for cloud security, governance, and cost optimization
• Troubleshoot production issues and perform root cause analysis
• Participate in architecture discussions, code reviews, and DevOps processes

Required Skills & Qualifications

• 4+ years of experience in Data Engineering
• Strong hands-on experience with Google Cloud Platform (GCP)
• Proficiency in Scala (mandatory)
• Solid experience with BigQuery and Dataflow / Apache Beam
• Strong understanding of distributed systems, data modeling, and performance tuning
• Experience building ETL/ELT pipelines for large-scale datasets
• Familiarity with SQL, NoSQL, and cloud-based data warehouses
• Knowledge of CI/CD pipelines, version control (Git), and Agile methodologies
• Strong problem-solving and communication skills

Good to Have (Preferred Skills)

• Experience with Apache Spark / Dataproc
• Exposure to real-time/streaming pipelines using Pub/Sub or Kafka
• Experience with Python alongside Scala
• Understanding of ML data pipelines and feature engineering
• GCP certifications (Professional Data Engineer preferred)

Preferred candidate profile