Senior Data Engineer - Ftc - Immediately Available

Afrikye; Durban; On-site; ZAR 600,000 - 900,000; Full time

Job summary

A leading company in the mining industry is seeking a skilled Data Engineer to develop data solutions utilizing Python and SQL. The role involves optimizing data pipelines, maintaining cloud services, and working with large datasets for analytics. The ideal candidate will have a strong data management background and be ready for an immediate start on a 6-month contract with potential for extension.

Qualifications

• Experience in cloud-based data platforms and agile environments.
• Strong background in data streaming and processing for mining analytics.
• Experience with version control (Git) and CI/CD.

Responsibilities

• Design, build, and maintain ETL / ELT pipelines for mining data processing.
• Develop and optimize SQL queries and data models for mining analysis.
• Collaborate with geologists and engineers to provide data insights.

Skills

Python

SQL

ETL / ELT

Data Warehousing

Big Data

GIS

Kafka

Education

BSc in engineering and/or sciences

MSc degree

Tools

Azure Data Factory

Databricks

PostgreSQL

MySQL

SQL Server

AWS

Job description

We are seeking a highly skilled Data Engineer with expertise in Python and SQL to develop and maintain data solutions for our client in the mining industry. The successful candidate will work with geologists, engineers, and data scientists to optimize data pipelines, process large datasets from IoT sensors and mining equipment, and support real-time analytics.

In order to be considered for this role, you must meet the following requirements:

• Matric
• BSc in engineering and/or sciences with demonstrable experience in machine learning. MSc degree is a plus
• Microsoft Certification in SQL Development

You must be an expert in:

Knowledge of ETL / ELT processes and tools

Experience designing, deploying code in cloud-based data platforms, and working in an agile environment

Programming languages: Python & SQL (advanced)

Experience setting up and maintaining Azure cloud services: Azure Data Factory, Blob Storage & Databricks

Big Data knowledge is a plus: Spark, Hadoop, or MapReduce

Experience in communication across systems (e.g., API)

Experience with sensor data ingestion and IoT data processing

Strong understanding of geospatial data processing (GIS, PostGIS, QGIS, or ArcGIS)

Knowledge of real-time data streaming technologies for monitoring mining operations

Ability to work with unstructured data from geological surveys, drone imagery, and seismic sensors

Strong background in data warehousing concepts for mining analytics and reporting

Experience with version control (Git) and CI/CD for data pipelines

Key Responsibilities (not limited to):

• Design, build, and maintain ETL / ELT pipelines to process data from mining equipment, IoT devices, and operational systems
• Develop and optimize SQL queries, stored procedures, and data models for efficient mining data analysis
• Write efficient Python scripts for data extraction, transformation, and integration from diverse sources
• Manage large-scale relational databases (PostgreSQL, MySQL, SQL Server) and NoSQL databases for unstructured geological and sensor data
• Work with real-time data streaming from mining equipment, GPS trackers, and environmental sensors using Kafka, Kinesis, or similar tools
• Ensure data quality, integrity, and governance for compliance with mining regulations and safety standards
• Collaborate with geologists, mining engineers, and operations teams to provide insights from data
• Implement predictive maintenance solutions by analyzing historical sensor and equipment data
• Work with GIS data and geospatial databases to support mine planning and exploration
• Deploy and maintain cloud-based data solutions (AWS, Azure, GCP) for scalable data storage and processing

This position is suitable for someone immediately available, on a 6-month contract with the possibility of extension.

If you meet all the above requirements, please send your CV to the provided contact.

Disclaimer: If you do not hear from us within 2 weeks, please consider your application unsuccessful.

Required Experience: Senior IC

Key Skills: Apache Hive, S3, Hadoop, Redshift, Spark, AWS, Apache Pig, NoSQL, Big Data, Data Warehouse, Kafka, Scala

Employment Type: Full-Time

Experience: years

Vacancy: 1