Senior Data Engineer: GCP Cloud-Native Pipelines Real-Time

PBT Group; Johannesburg; Hybrid; ZAR 1,200,000 - 1,800,000; Full time

Job summary

PBT Group is seeking a Senior Data Engineer in Johannesburg to design, develop, and optimize scalable data pipelines on Google Cloud Platform. The role focuses on cloud-native data platforms, streaming and batch processing, and near-real-time data pipelines.

The ideal candidate has extensive GCP experience, strong Python/SQL skills, and a track record in distributed data processing, with a blend of architecture and hands-on development.

Qualifications

• 8+ years of experience in Data Engineering.
• Strong practical experience with Google Cloud Platform (GCP) especially BigQuery, Dataflow, Pub/Sub, Cloud Run, GKE and Dataproc.
• Advanced Python and SQL skills.
• Experience designing and developing distributed data-processing solutions.

Responsibilities

• Design, develop and maintain scalable data pipelines and cloud-native data solutions on Google Cloud Platform.
• Build and optimise batch-processing and near-real-time data pipelines.
• Design and implement highly parallelised distributed data-processing solutions.

Skills

8+ years experience

Python

SQL

Education

Bachelor's degree in CS/IT/Engineering

Tools

GCP (BigQuery)

Dataflow

Pub/Sub

Cloud Run

GKE

Dataproc

Cloud Composer

Cloud Storage

dbt

Terraform

Job description

PBT Group is seeking a Senior Data Engineer in Johannesburg to design, develop, and optimize scalable data pipelines on Google Cloud Platform. The role focuses on cloud-native data platforms, streaming and batch processing, and near-real-time data pipelines.

The ideal candidate has extensive GCP experience, strong Python/SQL skills, and a track record in distributed data processing, with a blend of architecture and hands-on development.