Senior Data Engineer

PBT Group; Johannesburg; Hybrid; ZAR 1,200,000 - 1,800,000; Full time

Job summary

PBT Group is seeking a Senior Data Engineer in Johannesburg to design, develop, and optimize scalable data pipelines on Google Cloud Platform. The role focuses on cloud-native data platforms, streaming and batch processing, and near-real-time data pipelines.

The ideal candidate has extensive GCP experience, strong Python/SQL skills, and a track record in distributed data processing, with a blend of architecture and hands-on development.

Qualifications

• 8+ years of experience in Data Engineering.
• Strong practical experience with Google Cloud Platform (GCP) especially BigQuery, Dataflow, Pub/Sub, Cloud Run, GKE and Dataproc.
• Advanced Python and SQL skills.
• Experience designing and developing distributed data-processing solutions.

Responsibilities

• Design, develop and maintain scalable data pipelines and cloud-native data solutions on Google Cloud Platform.
• Build and optimise batch-processing and near-real-time data pipelines.
• Design and implement highly parallelised distributed data-processing solutions.

Skills

8+ years experience

Python

SQL

Education

Bachelor's degree in CS/IT/Engineering

Tools

GCP (BigQuery)

Dataflow

Pub/Sub

Cloud Run

GKE

Dataproc

Cloud Composer

Cloud Storage

dbt

Terraform

Job description

Senior Data Engineer job vacancy in Johannesburg.

PBT Group is seeking an experienced Senior Data Engineer with strong Google Cloud Platform (GCP) expertise to join a project focused on designing, developing and optimising scalable, cloud-native data platforms and near-real-time data pipelines.

The successful candidate will have extensive experience building highly parallelised, distributed data-processing solutions, with a strong understanding of batch and streaming architectures, event-driven processing and cloud-based data engineering.

Key Responsibilities:

• Design, develop and maintain scalable data pipelines and cloud-native data solutions on Google Cloud Platform.
• Build and optimise batch-processing and near-real-time data pipelines.
• Design and implement highly parallelised distributed data-processing solutions.
• Develop and manage workloads using Cloud Run and Google Kubernetes Engine (GKE).
• Design and implement streaming architectures using Pub/Sub and Dataflow.
• Work with large-scale datasets and optimise data processing for performance, scalability and cost efficiency.
• Implement and maintain data workflows and orchestration solutions.
• Collaborate with technical teams and stakeholders to deliver robust, reliable and scalable data solutions.

Minimum Requirements:

• 8+ years of experience in Data Engineering.
• Strong, practical experience with Google Cloud Platform (GCP), particularly:
  • BigQuery
  • Google Cloud Storage
  • Dataflow
  • Cloud Composer
  • Cloud Run
  • Google Kubernetes Engine (GKE)
  • Dataproc
  • Pub/Sub
• Strong experience with dbt.
• Advanced Python and SQL skills.
• Proven experience designing and developing distributed data-processing solutions.
• Strong understanding of highly parallelised processing, event-driven architectures and near-real-time data pipelines.
• Experience developing and optimising both batch and streaming data solutions.

Complementary Skills:

• Experience with any of the following will be advantageous:
• Microsoft Azure and/or AWS.
• Power BI and/or Looker.
• Apache Spark.
• Terraform and Infrastructure as Code.
• Microservices architectures.
• Event-driven architecture and integration patterns.

Qualifications:

• Relevant degree in Computer Science, Information Technology, Engineering or a related field.
• Google Cloud Professional Data Engineer certification would be advantageous.

Contract Details:

• Location: Johannesburg – Hybrid.
• Contract Duration: 12 months.
• Expected Start Date: December 2026 / January 2027.