Senior Data Engineer

Placements24; Riverton (UT); On-site; USD 36,000 - 73,000; Full time

Benefits offered by this job

Medical aid and retirement fund

Professional development

Annual performance bonus

Paid time off

Job summary

WhatJobs Direct is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and data warehouses in a data-driven environment. You will ensure data quality, security, and accessibility for data scientists and analysts, while enabling efficient analytics across large datasets.

The ideal candidate has strong programming skills (Python/SQL/Scala), hands-on experience with ETL/ELT, and familiarity with big data tools (Spark, Hadoop, Kafka).

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related field.

Responsibilities

• Design, build, and maintain scalable data pipelines (ETL/ELT).
• Develop and manage data warehouses and data lakes.
• Optimize data retrieval and processing performance.
• Ensure data quality and security across systems.
• Collaborate with data scientists and analysts to meet data needs.
• Implement and manage data integration from multiple sources.

Skills

Python

SQL

Scala

ETL/ELT

Spark

Hadoop

Kafka

PostgreSQL

MongoDB

Education

Bachelors in CS/Engineering/IT

Tools

Cloud platforms (AWS Redshift)

Azure Data Lake

Google BigQuery

Job description

About the Role WhatJobs Direct is seeking a skilled and experienced Senior Data Engineer to join our growing data analytics team in Upington, Northern Cape . This role is essential for designing, building, and maintaining our data pipelines and infrastructure, enabling efficient data warehousing and analytics capabilities. You will work with large, complex datasets, ensuring data quality, integrity, and accessibility for our data scientists and analysts. The ideal candidate possesses strong programming skills, deep knowledge of data warehousing concepts, ETL processes, and big data technologies. You will play a crucial role in shaping our data architecture and driving data-driven decision-making within the organization. This is an exciting opportunity to contribute to significant data initiatives in the Upington region and advance your career in a challenging yet rewarding environment. We foster a collaborative culture where innovation and continuous improvement are paramount.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines (ETL/ELT processes).
• Develop and manage data warehouses, data lakes, and other data storage solutions.
• Optimize data retrieval and processing performance.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists and analysts to understand their data needs and provide efficient data solutions.
• Implement and manage data integration from various sources.
• Automate data processes and workflows.
• Monitor data pipeline performance and troubleshoot issues proactively.
• Stay current with emerging data engineering technologies and best practices.
• Contribute to the design and implementation of the overall data architecture.
• Document data processes, schemas, and architecture.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field, or equivalent work experience.
• Minimum of 5 years of experience in data engineering or a related role.
• Strong programming skills in languages such as Python, SQL, or Scala.
• Proficiency with ETL/ELT tools and data integration techniques.
• Experience with relational databases (e.g., PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cassandra).
• Hands-on experience with big data technologies such as Apache Spark, Hadoop, or Kafka.
• Experience with cloud data platforms (e.g., AWS Redshift, Azure Data Lake, Google BigQuery) is highly desirable.
• Knowledge of data warehousing concepts and dimensional modeling.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.

Benefits

• Our client offers a competitive salary, comprehensive medical aid, and retirement fund contributions.
• We provide excellent opportunities for professional development through training, certifications, and attendance at relevant industry events.
• This role is based in our Upington office, providing a stable work environment.
• Additional benefits include annual performance bonuses, generous paid time off, and employee assistance programs.
• We are committed to fostering a positive work-life balance and offering clear career advancement opportunities within the data domain.
• Our client invests in employee growth and well-being.