Data Engineer: Scalable Pipelines & Real-Time (Hybrid)

AES Global; Johannesburg; Hybrid; ZAR 700,000 - 1,000,000; Full time

Job summary

AES Global in Johannesburg is seeking an Intermediate Data Engineer to build and support scalable data pipelines, handling large data volumes in batch and real time. You will create data feeds into AWS and develop for BI with automated testing and deployment.

The role requires 3–5 years in data/soft engineering, strong Python and PySpark, data modelling, and a bachelor’s degree in CS or related field.

Qualifications

• 3–5 years’ data-engineering or software-engineering experience, including big-data and ETL work.
• Strong Python and PySpark or Spark.
• Big-data batch and streaming tools, and AWS (EMR, EC2, S3).

Responsibilities

• Build and support scalable, secure data pipelines from diverse sources.
• Handle large data volumes in batch and real time.
• Create data marts using Talend or similar ETL tools; manipulate data with Python.
• Process data using Hadoop paradigms (e.g., EMR) and create data feeds into AWS Cloud.
• Develop for big data and BI with automated testing and deployment.
• Translate requirements into performant data solutions.

Skills

Python

PySpark

Big data

Data modelling

Education

Bachelor's degree in Computer Science or related field

Tools

Talend

AWS (EMR, S3, EC2)

Job description

AES Global in Johannesburg is seeking an Intermediate Data Engineer to build and support scalable data pipelines, handling large data volumes in batch and real time. You will create data feeds into AWS and develop for BI with automated testing and deployment.

The role requires 3–5 years in data/soft engineering, strong Python and PySpark, data modelling, and a bachelor’s degree in CS or related field.