Data Engineer: Build Scalable Cloud Pipelines

Network Finance; Centurion; On-site; ZAR 600,000 - 900,000; Full time

Job summary

Network Finance in Centurion (South Africa) seeks a data engineer to shape data architecture, build scalable pipelines, and drive innovative data solutions. You will work with AWS/Azure/GCP, Kafka, Spark, Python, and SQL to transform complex data into actionable insights.

The role focuses on designing end-to-end data architectures, handling large datasets, and integrating data from multiple sources while collaborating with a team of engineers and data scientists.

Qualifications

• Competed BSC in IT, Mathematics or a related field.
• 3+ years of working as a Data Engineerdesigning and building robust, scalable data pipelines. You're proficient in Python, SQL, ETL processes, and data wrangling techniques.
• Cloud Mastery: Experience working with major cloud platforms such as AWS, Azure, or Google Cloud. You're familiar with services like S3, Redshift, BigQuery, EMR, and Dataflow.
• Big Data Knowledge: You have hands-on experience with distributed data systems like Apache Kafka, Spark, Hadoop, or Flink. You are comfortable managing massive datasets and streamlining data processing workflows.
• Database Expertise: Whether it's relational (PostgreSQL, MySQL) or NoSQL (Cassandra, MongoDB), you understand the nuances of selecting, optimizing, and managing databases to meet various data storage needs.
• Data Architecture Visionary: You have experience designing end-to-end data architecture, ensuring scalability, reliability, and efficiency in data pipelines, and integrating data from multiple sources seamlessly.

Skills

Python

SQL

ETL

Data wrangling

Data architecture

Cloud platforms

Kafka

Spark

Hadoop

Flink

PostgreSQL

MySQL

Cassandra

MongoDB

Education

BSc in IT, Mathematics or related field

Tools

Kafka

Spark

Hadoop

Flink

PostgreSQL

MySQL

Cassandra

MongoDB

Job description

Network Finance in Centurion (South Africa) seeks a data engineer to shape data architecture, build scalable pipelines, and drive innovative data solutions. You will work with AWS/Azure/GCP, Kafka, Spark, Python, and SQL to transform complex data into actionable insights.

The role focuses on designing end-to-end data architectures, handling large datasets, and integrating data from multiple sources while collaborating with a team of engineers and data scientists.