Hybrid Senior Data Engineer - Big Data & ETL Pipelines

Placements24; Sandton; Hybrid; ZAR 900,000 - 1,600,000; Full time

Benefits offered by this job

Competitive salary

Hybrid work arrangement

Professional development opportunities

Challenging data projects

Collaborative environment

Job summary

Placements24 client, a dynamic data analytics firm in Sandton, Gauteng, is seeking a Senior Data Engineer to design, build and maintain scalable data pipelines and infrastructure for BI, ML and reporting initiatives.

You will work with ETL/ELT pipelines, big data technologies (Spark, Hadoop), and cloud platforms (AWS, Azure, GCP), collaborating with data scientists and business analysts to deliver reliable data solutions. Hybrid work model offered.

Qualifications

• Minimum 5 years of experience in data engineering or a related field.
• Proven experience designing and implementing complex ETL/ELT pipelines.
• Strong proficiency in SQL and experience with relational databases (PostgreSQL, SQL Server).
• Hands-on experience with Spark and distributed systems.
• Experience with cloud data platforms (AWS Redshift, S3, Glue; Azure Data Factory, Synapse; GCP BigQuery, Dataflow).
• Proficiency in Python or Scala for data manipulation and pipeline development.
• Familiarity with data warehousing concepts and best practices.
• Experience with data modeling and schema design.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.

Responsibilities

• Design, develop, and optimize robust ETL/ELT data pipelines to ingest, transform, and load data into data warehouses and data lakes.
• Build and maintain scalable data processing systems using big data technologies (e.g., Apache Spark, Hadoop).
• Develop and manage data models and database schemas for optimal performance and usability.
• Implement data quality checks and validation processes to ensure data accuracy and consistency.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver data solutions.
• Monitor data pipeline performance, troubleshoot issues, and implement solutions for improved efficiency and reliability.
• Develop and maintain data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
• Stay current with emerging data engineering technologies and best practices.
• Document data architectures, pipelines, and processes.

Skills

SQL

PostgreSQL

SQL Server

Apache Spark

Hadoop

Python

Scala

Data warehousing

Data modeling

ETL/ELT

Cloud platforms

Problem solving

Communication

Tools

Spark/Hadoop

AWS Redshift/S3/Glue

Azure Data Factory/Synapse

GCP BigQuery/Dataflow

Job description

Placements24 client, a dynamic data analytics firm in Sandton, Gauteng, is seeking a Senior Data Engineer to design, build and maintain scalable data pipelines and infrastructure for BI, ML and reporting initiatives.

You will work with ETL/ELT pipelines, big data technologies (Spark, Hadoop), and cloud platforms (AWS, Azure, GCP), collaborating with data scientists and business analysts to deliver reliable data solutions. Hybrid work model offered.