Data Engineer — Real-Time Analytics & ML Pipelines

Boardroom Appointments; Durban; On-site; ZAR 600,000 - 800,000; Full time

Job summary

A recruitment agency is seeking an Intermediate Data Engineer in Durban, South Africa. The successful candidate will have a strong background in Scala or Java, experience with cloud platforms like AWS, GCP, or Azure, and a solid understanding of data modeling principles. Responsibilities include building scalable data infrastructure, developing APIs for data science outputs, and optimizing cloud costs. The ideal candidate will possess a Master's in a related field and have 2-3 years of relevant experience.

Qualifications

• Matric (Grade 12) is required.
• 2-3 years of relevant work experience is a must.
• Knowledge of API development and machine learning deployment.

Responsibilities

• Build and scale data infrastructure for real-time data processing.
• Create scalable data ingestion and machine learning inference pipelines.
• Develop APIs to deliver data science outputs.
• Scale production systems for increased demand.
• Provide visibility into data platform health.
• Automate and handle life-cycle of data processing systems.

Skills

Scala

Java

AWS

GCP

Azure

Data modeling

Relational databases

NoSQL databases

Spark

Hadoop

Kafka

Docker

Kubernetes

Education

Masters in Software Engineering, Data Engineering, Computer Science or related field

Tools

Postgres

ElasticSearch/OpenSearch

Graph databases

Job description

A recruitment agency is seeking an Intermediate Data Engineer in Durban, South Africa. The successful candidate will have a strong background in Scala or Java, experience with cloud platforms like AWS, GCP, or Azure, and a solid understanding of data modeling principles. Responsibilities include building scalable data infrastructure, developing APIs for data science outputs, and optimizing cloud costs. The ideal candidate will possess a Master's in a related field and have 2-3 years of relevant experience.