Google Cloud Platform (GCP) Data Engineer

PBT Group; Johannesburg; On-site; ZAR 900,000 - 1,300,000; Full time

Job summary

PBT Group is seeking a Google Cloud Platform (GCP) Data Engineer to design, build, and optimize data pipelines and architectures on GCP. You will collaborate with data analysts, data scientists, and stakeholders to ensure high-quality data for analytics, ML, and real-time processing.

You will implement ETL/ELT pipelines, manage data warehouses and data lakes with BigQuery, Cloud Storage, Dataflow, and Cloud Composer (Airflow), and contribute to data modelling, governance, and CI/CD automation

Qualifications

• Bachelor’s degree in a related field (CS/IS/Engineering).
• 4+ years of data engineering experience, with at least 2 years on GCP.
• Strong SQL and ETL/ELT capabilities for large-scale data.
• Understanding of data warehousing and dimensional modelling.
• Hands-on with BigQuery or Snowflake and orchestration tools like Airflow or Argo.
• Experience with real-time data processing using Kafka Streaming.
• Knowledge of ML pipelines and MLOps is a plus.

Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines on GCP.
• Implement and manage data warehouses and data lakes using BigQuery, Cloud Storage, Dataflow, and Cloud Composer (Airflow).
• Develop and optimise data models for business needs, ensuring performance and scalability.
• Collaborate with cross-functional teams to deliver tailored data access solutions.
• Ensure data quality, security, and governance through tuning and capacity management.
• Implement CI/CD automation using GitHub Actions and Docker to streamline deployments.
• Develop and maintain documentation for data architecture and processes.
• Support real-time data processing with Kafka Streaming and ML workflows.

Skills

Data engineering

SQL

ETL/ELT

GCP

BigQuery

Airflow

Python

Docker

Kubernetes

GitHub Actions

Data modelling

Kafka Streaming

MLOps

Education

Bachelor’s Degree in Computer Science / Information Systems / Engineering

Tools

BigQuery

Snowflake

Airflow

Argo

Medallion Architecture

Job description

Our team is seeking an experienced Google Cloud Platform (GCP) Data Engineer with a strong background in building and optimising data pipelines, architectures, and data sets on GCP. This role involves close collaboration with data analysts, data scientists, and other stakeholders to ensure high-quality data availability for advanced analytics, machine learning, and real-time data processing.

Key Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines on GCP to support analytics, reporting, and machine learning applications.

• Implement and manage data warehouses and data lakes using GCP services such as BigQuery, Cloud Storage, Cloud Dataflow, and Cloud Composer (Airflow).

• Develop and optimise data models to align with business needs, ensuring scalability and performance.

• Collaborate with cross-functional teams to understand business requirements and deliver tailored data access solutions.

• Ensure data quality, security, and governance through performance tuning, capacity management, and compliance best practices.

• Implement CI/CD automation using GitHub Actions and Docker to streamline deployment of data pipelines.

• Develop and maintain documentation for data architecture, models, and processes.

• Support real-time data processing with tools such as Kafka Streaming and MLOps workflows for machine learning pipelines.

Qualifications

• Bachelor’s Degree in Computer Science, Information Systems, Engineering, or a related field.

• 4+ years of experience in data engineering, with at least 2 years focused on GCP.

• Strong experience with SQL and ETL/ELT processes for transforming and processing large-scale structured and unstructured data.

• Solid understanding of data warehousing concepts and dimensional modelling techniques.

• Hands‑on experience with BigQuery or Snowflake, as well as orchestration tools like Airflow or Argo.

• Experience working with real-time data processing using Kafka Streaming.

• Knowledge of machine learning pipelines and MLOps practices is a plus.

Technical Skills

• Google Cloud Platform (GCP) – BigQuery, Cloud Dataflow, Cloud Storage, Cloud Composer.

• SQL & DBT – Data modelling and transformation within Medallion Architecture.

• Python – Development of data workflows, automation, and integration.

• Kafka Streaming – Real-time data processing.

• GitHub & CI/CD (GitHub Actions) – Version control and automated deployment.

• Docker & Kubernetes (GKE) – Containerisation and orchestration.

• Airflow (Cloud Composer) – Data pipeline orchestration.

Preferred Competencies

• Google Cloud Certification (e.g., Professional Data Engineer) is a plus.

• Experience with data governance and data quality frameworks.

• Strong problem‑solving skills and ability to work in a fast‑paced, collaborative environment.

• Familiarity with machine learning workflows and MLOps integration.