Senior Data Engineer: GCP & GenAI Streaming Pipelines

ADG; Johannesburg; Sur place; ZAR 900 000 - 1 500 000; Plein temps

Résumé du poste

ADG in Johannesburg seeks an experienced Data Engineer to join our Data and AI practice. You will design, build, and lead robust batch and streaming data pipelines on GCP and open-source ecosystems, delivering value to clients and our business.

You will drive project delivery, mentor junior engineers, and apply GenAI concepts to real-world data challenges, ensuring scalable, reliable solutions across the data stack.

Qualifications

• 7+ years of proven industry experience in data engineering, data architecture, and enterprise software systems.
• A Computer Science (or equivalent) degree, or equivalent practical experience.
• Deep expertise in modern cloud architectures and GCP data engineering services for designing resilient batch and streaming pipelines.
• Exceptional, hands-on proficiency in Python, SQL, and Apache Spark, as well as data management concepts.
• Deep understanding of diverse data architectures, Hadoop, distributed computing, cloud-native databases, data lakehouses, and analytics.
• Strong grasp of Generative AI (GenAI) technologies and their business applications.

Responsabilités

• Lead the design and delivery of robust batch and real-time data pipelines on GCP and open-source ecosystems.
• Drive technical accountability, timelines, and high-quality outcomes aligned with client goals.
• Mentor junior engineers and guide technical discussions and POC development.
• Translate complex architectures into business value and opportunities for service improvements.

Connaissances

Python

SQL

Apache Spark

Formation

Bachelor's degree in Computer Science or equivalent

Outils

BigQuery

Cloud Dataflow

Cloud Storage

Pub/Sub

Cloud Dataproc

Cloud Composer

Dataplex

Description du poste

ADG in Johannesburg seeks an experienced Data Engineer to join our Data and AI practice. You will design, build, and lead robust batch and streaming data pipelines on GCP and open-source ecosystems, delivering value to clients and our business.

You will drive project delivery, mentor junior engineers, and apply GenAI concepts to real-world data challenges, ensuring scalable, reliable solutions across the data stack.