Senior Data Engineer

Placements24; Upington; On-site; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Medical aid

Retirement fund contributions

Training and certifications

Annual performance bonuses

Generous paid time off

Employee assistance programs

Job summary

WhatJobs Direct is seeking a skilled and experienced Senior Data Engineer to join our growing data analytics team in Upington, Northern Cape. This role focuses on designing, building, and maintaining data pipelines and infrastructure to enable efficient data warehousing and analytics for data scientists and analysts.

The ideal candidate will have 5+ years in data engineering, strong programming skills in Python/SQL/Scala, and hands-on experience with Spark, Hadoop, Kafka, and cloud platforms

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field, or equivalent work experience.
• Minimum of 5 years of experience in data engineering or a related role.
• Strong programming skills in Python, SQL, or Scala.
• Proficiency with ETL/ELT tools and data integration techniques.
• Experience with relational databases (e.g., PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cassandra).
• Hands-on experience with big data technologies such as Apache Spark, Hadoop, or Kafka.
• Experience with cloud data platforms (e.g., AWS Redshift, Azure Data Lake, Google BigQuery) is highly desirable.
• Knowledge of data warehousing concepts and dimensional modeling.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines (ETL/ELT processes).
• Develop and manage data warehouses, data lakes, and other data storage solutions.
• Optimize data retrieval and processing performance.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists and analysts to understand their data needs and provide efficient data solutions.
• Implement and manage data integration from various sources.
• Automate data processes and workflows.
• Monitor data pipeline performance and troubleshoot issues proactively.
• Stay current with emerging data engineering technologies and best practices.
• Contribute to the design and implementation of the overall data architecture.
• Document data processes, schemas, and architecture.

Skills

Python

SQL

Scala

ETL/ELT concepts

Data integration

Data modeling

Big data concepts

Education

Bachelor's degree in Computer Science, Engineering, Information Technology, or related field

Tools

PostgreSQL

SQL Server

MongoDB

Cassandra

Apache Spark

Hadoop

Kafka

AWS Redshift

Azure Data Lake

Google BigQuery

Job description

About the Role

WhatJobs Direct is seeking a skilled and experienced Senior Data Engineer to join our growing data analytics team in Upington, Northern Cape. This role is essential for designing, building, and maintaining our data pipelines and infrastructure, enabling efficient data warehousing and analytics capabilities. You will work with large, complex datasets, ensuring data quality, integrity, and accessibility for our data scientists and analysts. The ideal candidate possesses strong programming skills, deep knowledge of data warehousing concepts, ETL processes, and big data technologies. You will play a crucial role in shaping our data architecture and driving data-driven decision‑making within the organization. This is an exciting opportunity to contribute to significant data initiatives in the Upington region and advance your career in a challenging yet rewarding environment. We foster a collaborative culture where innovation and continuous improvement are paramount.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines (ETL/ELT processes).


• Develop and manage data warehouses, data lakes, and other data storage solutions.


• Optimize data retrieval and processing performance.


• Ensure data quality, integrity, and security across all data systems.


• Collaborate with data scientists and analysts to understand their data needs and provide efficient data solutions.


• Implement and manage data integration from various sources.


• Automate data processes and workflows.


• Monitor data pipeline performance and troubleshoot issues proactively.


• Stay current with emerging data engineering technologies and best practices.


• Contribute to the design and implementation of the overall data architecture.


• Document data processes, schemas, and architecture.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field, or equivalent work experience.


• Minimum of 5 years of experience in data engineering or a related role.


• Strong programming skills in languages such as Python, SQL, or Scala.


• Proficiency with ETL/ELT tools and data integration techniques.


• Experience with relational databases (e.g., PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cassandra).


• Hands‑on experience with big data technologies such as Apache Spark, Hadoop, or Kafka.


• Experience with cloud data platforms (e.g., AWS Redshift, Azure Data Lake, Google BigQuery) is highly desirable.


• Knowledge of data warehousing concepts and dimensional modeling.


• Strong analytical and problem‑solving skills.


• Excellent communication and teamwork abilities.

Benefits

• Our client offers a competitive salary, comprehensive medical aid, and retirement fund contributions.


• We provide excellent opportunities for professional development through training, certifications, and attendance at relevant industry events.


• This role is based in our Upington office, providing a stable work environment.


• Additional benefits include annual performance bonuses, generous paid time off, and employee assistance programs.


• We are committed to fostering a positive work‑life balance and offering clear career advancement opportunities within the data domain.


• Our client invests in employee growth and well‑being.