Senior Data Platform Engineer - TB-scale, Hybrid

Impact; Cape Town; Hybride; ZAR 900 000 - 1 300 000; Plein temps

Résumé du poste

impact.com seeks a Data Platform Engineer in Cape Town to own TB-scale pipelines and platform components. You will design scalable batch and streaming ETL/ELT jobs, ensure SLOs, reliability, and cost efficiency, and participate in on-call as Tier 2.

Within the Data Analytics Group, you’ll collaborate with Analytics Engineers, Data Analysts, Data Scientists, and Data Product Managers while mentoring newer engineers and shaping governance, observability, and self-service capabilities across the

Qualifications

• 3 to 5 years of experience in data platform, data engineering, or backend/distributed systems engineering.
• Strong Python, plus production experience in a JVM language, with proper error handling, logging, and testing.
• Scala is preferred as the primary pipeline language; willingness to deepen.
• Hands-on depth in distributed batch processing (Spark) or streaming (Kafka/ Pub/Sub).
• Advanced SQL including window functions and optimisation.
• Experience with Airflow: scheduling, dependencies, retry logic, and coordination.
• Working fluency with a major cloud platform, ideally GCP (BigQuery, Dataproc, Cloud Storage, Pub/Sub).
• Solid engineering practices: Git, CI/CD, automated testing, code review, monitoring, and on-call readiness.

Responsabilités

• Own platform components end-to-end, including design, implementation, reliability, and cost.
• Design and build scalable ETL/ELT and streaming pipelines using Spark/Dataproc, Kafka, and Pub/Sub.
• Integrate diverse sources into BigQuery and other stores.
• Build modular, fault-tolerant components with schema evolution.
• Configure and maintain core platform infrastructure as code (Dataproc, Kafka, Airflow, BigQuery).
• Tune performance and cost through query optimisation and partitioning; optimise streaming configurations.
• Own and track SLOs for freshness, correctness, and latency; implement monitoring and data quality checks.
• Mentor Associate Data Platform Engineers through pairing and code reviews; communicate platform trade-offs clearly.

Connaissances

Python

Scala

SQL

Git

CI/CD

Distributed systems

On-call

Data quality

Outils

Spark

Dataproc

BigQuery

Kafka

Pub/Sub

Airflow

Astronomer

GCP

Description du poste

impact.com seeks a Data Platform Engineer in Cape Town to own TB-scale pipelines and platform components. You will design scalable batch and streaming ETL/ELT jobs, ensure SLOs, reliability, and cost efficiency, and participate in on-call as Tier 2.

Within the Data Analytics Group, you’ll collaborate with Analytics Engineers, Data Analysts, Data Scientists, and Data Product Managers while mentoring newer engineers and shaping governance, observability, and self-service capabilities across the