Senior Data Engineer – Google Cloud Platform (GCP) (Contract)

PBT Group; Johannesburg; Hybrid; ZAR 900,000 - 1,500,000; Full time

Job summary

PBT Group is seeking an experienced Senior Data Engineer with strong Google Cloud Platform (GCP) expertise to join a project focused on designing, developing and optimising scalable, cloud-native data platforms and near-real-time data pipelines.

The successful candidate will have extensive experience building highly parallelised, distributed data-processing solutions, with a strong understanding of batch and streaming architectures, event-driven processing and cloud-based data engineering.

Qualifications

• 8+ years of experience in Data Engineering.
• Strong GCP experience with BigQuery, Dataflow, Cloud Composer, and GKE.
• Proficient in Python and SQL.
• Experience designing distributed, scalable data pipelines (batch and streaming).
• Experience with dbt and cloud-native architectures.

Responsibilities

• Design, develop and maintain scalable cloud-native data pipelines on GCP.
• Build and optimise batch and near-real-time data pipelines.
• Implement distributed data-processing solutions with parallelism.
• Develop workloads using Cloud Run and GKE.
• Design streaming architectures with Pub/Sub and Dataflow.
• Work with large-scale datasets to optimise performance and cost.
• Maintain data workflows and orchestration solutions.
• Collaborate with technical teams and stakeholders to deliver robust data solutions.

Skills

Data engineering

GCP expertise

Python

SQL

Distributed processing

Batch & streaming

Education

Relevant degree in CS/IT/Engineering

Google Cloud Professional Data Engineer certification (advantageous)

Tools

BigQuery

Dataflow

Cloud Run

GKE

Pub/Sub

Cloud Composer

Dataproc

Job description

PBT Group is seeking an experienced Senior Data Engineer with strong Google Cloud Platform (GCP) expertise to join a project focused on designing, developing and optimising scalable, cloud-native data platforms and near-real-time data pipelines.

The successful candidate will have extensive experience building highly parallelised, distributed data-processing solutions, with a strong understanding of batch and streaming architectures, event-driven processing and cloud-based data engineering.

Key Responsibilities

• Design, develop and maintain scalable data pipelines and cloud-native data solutions on Google Cloud Platform.

• Build and optimise batch-processing and near-real-time data pipelines.

• Design and implement highly parallelised distributed data-processing solutions.

• Develop and manage workloads using Cloud Run and Google Kubernetes Engine (GKE).

• Design and implement streaming architectures using Pub/Sub and Dataflow.

• Work with large-scale datasets and optimise data processing for performance, scalability and cost efficiency.

• Implement and maintain data workflows and orchestration solutions.

• Collaborate with technical teams and stakeholders to deliver robust, reliable and scalable data solutions.

Required Skills and Experience

• 8+ years of experience in Data Engineering.

• Strong, practical experience with Google Cloud Platform (GCP), particularly:

  • BigQuery

  • Google Cloud Storage

  • Dataflow

  • Cloud Composer

  • Cloud Run

  • Google Kubernetes Engine (GKE)

  • Dataproc

  • Pub/Sub

• Strong experience with dbt.

• Advanced Python and SQL skills.

• Proven experience designing and developing distributed data-processing solutions.

• Strong understanding of highly parallelised processing, event-driven architectures and near-real-time data pipelines.

• Experience developing and optimising both batch and streaming data solutions.

Complementary Skills

Experience with any of the following will be advantageous:

• Microsoft Azure and/or AWS.

• Power BI and/or Looker.

• Apache Spark.

• Terraform and Infrastructure as Code.

• Microservices architectures.

• Event-driven architecture and integration patterns.

Qualifications

• Relevant degree in Computer Science, Information Technology, Engineering or a related field.

• Google Cloud Professional Data Engineer certification would be advantageous.

Contract Details

• Location: Johannesburg – Hybrid

• Contract Duration: 12 months

• Expected Start Date: December 2026 / January 2027