Data Engineer – Big Data Platforms (Upington)

Placements24; Siyancuma Local Municipality; Hybrid; ZAR 550,000 - 750,000; Full time

Job summary

Placements24 is seeking a Data Engineer for Upington to design, build and maintain scalable data pipelines and platforms, enabling efficient data processing, storage and analysis. You will ensure data availability and integrity to support data scientists and analysts.

You will work with big data technologies, develop data warehouses and data lakes, monitor performance, and collaborate with stakeholders to deliver data solutions.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• 3+ years of experience as a Data Engineer or in a similar role involving big data technologies.
• Proficiency with distributed data processing frameworks such as Spark, Hadoop, or Flink.
• Experience with SQL and NoSQL databases, as well as data warehousing concepts.
• Knowledge of cloud platforms (AWS, Azure, GCP) and their data services is a plus.
• Strong programming skills in Python, Scala, or Java.

Responsibilities

• Design, build, and optimize robust and scalable data pipelines for ingesting, transforming, and processing large volumes of data.
• Develop and maintain data warehousing solutions and data lakes using modern big data technologies.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Monitor data infrastructure performance, troubleshoot issues, and implement optimizations.
• Stay current with emerging big data technologies and best practices.

Skills

Python

Scala

Java

SQL

NoSQL

Spark

Hadoop

Cloud platforms

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

AWS

Azure

GCP

Job description

Placements24 is seeking a Data Engineer for Upington to design, build and maintain scalable data pipelines and platforms, enabling efficient data processing, storage and analysis. You will ensure data availability and integrity to support data scientists and analysts.

You will work with big data technologies, develop data warehouses and data lakes, monitor performance, and collaborate with stakeholders to deliver data solutions.