Data Engineer (Dataworks)

e-Merge IT Recruitment; Johannesburg; Hybrid; ZAR 900,000 - 1,100,000; Full time

Job summary

e-Merge IT Recruitment seeks an experienced Data Engineer to join their DataWorks team in Johannesburg. You will design, build, and optimise data pipelines and cloud-based platforms to enable BI, analytics, and ML initiatives.

Key tech includes AWS, Microsoft Fabric, and Databricks, with exposure to Spark, Hadoop, and Kafka. A strong background in Python, Pandas, and SQL is required.

Qualifications

• Bachelor’s degree in a related field is required.
• 7+ years of hands-on data engineering or platform experience.
• Experience building scalable data pipelines and data solutions.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Build cloud-based data solutions using AWS, Fabric, or Databricks.
• Integrate data from multiple sources into data platforms, lakes, or warehouses.
• Develop data models, data marts, and warehouse structures for BI and analytics.
• Ensure data quality, integrity, and reliability with monitoring and validation.
• Work with large datasets and big data tech like Spark to transform data efficiently.
• Collaborate with data analysts, BI developers, data scientists, and stakeholders.
• Optimize data workflows to improve performance and scalability.

Skills

7+ years experience

Data pipelines

ETL/ELT

Python

Pandas

SQL

Education

Bachelor’s degree in Computer Science/Engineering/IT or related

Tools

AWS Glue

Redshift

S3

Lambda

EMR

Athena

Microsoft Fabric

Databricks

Apache Spark

Hadoop

Kafka

Job description

Data Engineer (Dataworks) – Johannesburg – up to R1mil per annum

Our client is looking for an experienced Data Engineer to join their DataWorks team and play a key role in designing, developing, and delivering innovative and scalable data solutions. The successful candidate will be responsible for building and optimising data pipelines, developing cloud-based data platforms, and managing data infrastructure that enables business intelligence, advanced analytics, and machine learning initiatives.

This role requires strong technical expertise across Microsoft Fabric, AWS, and Databricks, as well as proven experience integrating data from multiple systems and sources.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes to support analytics and reporting requirements.
• Build and manage cloud-based data solutions using platforms such as AWS, Microsoft Fabric, or Databricks.
• Integrate data from multiple internal and external sources into centralized data platforms, data lakes, or data warehouses.
• Develop and maintain data models, data marts, and data warehouse structures to support business intelligence and analytics initiatives.
• Ensure data quality, integrity, and reliability through validation, monitoring, and automated quality checks.
• Work with large datasets and big data technologies such as Spark to process and transform complex data efficiently.
• Collaborate closely with data analysts, BI developers, data scientists, and business stakeholders to understand data requirements and deliver scalable solutions.
• Optimize data processing workflows and improve performance and scalability of existing data pipelines.

Qualifications & Experience:

• Bachelor’s degree from an accredited university. A degree in Computer Science, Engineering, Information Technology, or a related field is advantageous.
• Minimum of 7+ years of hands‑on experience in data engineering or data platform development. Experience within the financial services industry would be beneficial but is not essential.
• Proven experience building, managing, and optimizing data pipelines and data solutions using on-premise or cloud-based technologies.
• Practical experience working with AWS services such as Glue, Redshift, S3, Lambda, EMR, and Athena, or exposure to Microsoft Fabric or Databricks.
• Familiarity with big data frameworks and technologies such as Apache Spark, Hadoop, or Kafka.
• Strong programming capabilities in Python, Pandas, and SQL for data processing and transformation

The Reference Number for this position is NG60984 which is a Permanent, Hybrid role in Johannesburg offering a salary of up to R1mil per annum negotiable based on experience.