Senior Big Data Engineer — Scalable Pipelines (Hybrid)

Placements24; Pretoria; Hybrid; ZAR 900,000 - 1,500,000; Full time

Benefits offered by this job

Hybrid work model

Professional development opportunities

Conferences and training

Job summary

WhatJobs Direct is seeking a Senior Data Engineer in Pretoria, Gauteng. You will build scalable data pipelines using Spark/Hadoop, develop ETL/ELT processes, and manage data warehouses and lakes to support analytics.

Role requires strong SQL, data warehousing, and cloud experience with AWS or similar platforms. Collaboration with data scientists and engineers is essential in a hybrid work environment.

Qualifications

• Degree in a quantitative field and proven data engineering experience.
• Strong background with big data technologies and data pipelines.
• Proficiency in Python, Scala, or Java for ETL/ELT development.
• Hands-on with Spark, Hadoop ecosystem, and Kafka or Flink.

Responsibilities

• Design, build, and maintain scalable data pipelines using Spark/Hadoop.
• Develop and optimize ETL/ELT processes for ingestion and transformation.
• Build data warehouses and data lakes for reporting and analytics.
• Ensure data quality, integrity, and accessibility across platforms.
• Collaborate with data scientists and analysts to meet data needs.
• Implement data governance policies and monitor performance.

Skills

Python/Scala/Java

Big data

SQL

Data warehousing

Data governance

Collaboration

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Spark

Hadoop

Kafka

Hive

AWS EMR

Job description

WhatJobs Direct is seeking a Senior Data Engineer in Pretoria, Gauteng. You will build scalable data pipelines using Spark/Hadoop, develop ETL/ELT processes, and manage data warehouses and lakes to support analytics.

Role requires strong SQL, data warehousing, and cloud experience with AWS or similar platforms. Collaboration with data scientists and engineers is essential in a hybrid work environment.