Senior Data Engineer

AES Global; Johannesburg; Hybrid; ZAR 900,000 - 1,500,000; Full time

Job summary

AES Global is seeking a Senior Data Engineer to architect, build and lead scalable data solutions as part of an agile team. You’ll turn complex requirements into high-performing data architecture and build the infrastructure that turns raw data into insight at scale, in batch and real time.

You’ll act as a technical lead—translating requirements into architecture, leading analytics across projects, and creating data marts with ETL tools using Python, Hadoop (EMR), and AWS cloud feeds.

Qualifications

• Bachelor's degree in CS/CE or equivalent required.
• AWS certification or equivalent cloud-certification desirable.
• 6+ years of data-engineering or software-engineering experience, with 2+ years in big data.

Responsibilities

• Architect the data-analytics framework and lead batch and real-time analytical solutions.
• Translate complex functional and technical requirements into detailed architecture, design and high-performing software.
• Lead user-story analysis, design, development, testing and build-automation across multiple projects.
• Create data marts using Talend or a similar ETL tool, and manipulate data using Python.
• Process data using the Hadoop paradigm (e.g. EMR), and create and support data feeds into AWS Cloud.
• Develop for big data and business intelligence, including automated testing and deployment.

Skills

Data engineering

Python

PySpark

Spark

AWS

ETL

Hadoop

Big data

Data modeling

OO design

Education

Bachelor's degree in CS/CE or equivalent

AWS certification

Tools

Talend

Hadoop

EMR

BigQuery

DBT

AWS

GCP/Azure

Job description

We’re working with an established technology consultancy, and they’re looking for a Senior Data Engineer to architect, build and lead scalable, secure data solutions as part of an agile team.

You’ll act as a technical lead — turning complex requirements into high-performing data architecture, and building the infrastructure that turns raw data into insight at scale, in batch and real time.

What you’ll do

• Architect the data-analytics framework and lead batch and real-time analytical solutions
• Translate complex functional and technical requirements into detailed architecture, design and high-performing software
• Lead user-story analysis, design, development, testing and build-automation across multiple projects
• Create data marts using Talend or a similar ETL tool, and manipulate data using Python
• Process data using the Hadoop paradigm (e.g. EMR), and create and support data feeds into AWS Cloud
• Develop for big data and business intelligence, including automated testing and deployment

What you’ll bring

• 6+ years’ data-engineering or software-engineering experience, with 2+ years in big data
• 2+ years with ETL processes and 2+ years’ AWS experience
• Strong Python, and PySpark or Spark
• The ability to architect highly scalable distributed systems using open-source tools
• Expert data modelling, and strong object-oriented design, coding and testing patterns
• A bachelor’s degree in Computer Science, Computer Engineering or equivalent, and AWS certification

Advantageous

• BigQuery, DBT, and a combination of AWS / GCP / Azure
• AI tooling (e.g. Bedrock or similar)

Working model

Hybrid in Johannesburg — three days a week in the office.