Senior Big Data Engineer - Remote, Cloud & Analytics

Placements24; Klerksdorp; Ibrido; ZAR 900.000 - 1.500.000; Tempo pieno

Vantaggi offerti da questo lavoro

Fully remote work

Flexible hours

Home office setup allowance

Professional development

Conferences

Generous paid time off

Retirement fund participation

Descrizione del lavoro

Placements24 is seeking a Senior Data Engineer in South Africa with a strong background in big data and analytics. This remote-first role involves designing, building, and maintaining scalable data pipelines and data lakes to support advanced analytics and decision-making.

You will work with Spark, Hadoop, and cloud platforms to ensure data quality, security, and performance, collaborating with data scientists and business stakeholders in a distributed team environment.

Competenze

• Minimum of 5 years of data engineering experience with big data technologies.
• Proficiency in Python, Scala or Java.
• Hands-on experience with distributed data processing frameworks like Spark and Hadoop ecosystem.
• Experience with cloud data platforms such as AWS, Azure or GCP.
• Strong SQL skills and experience with relational and NoSQL databases.
• Understanding of data warehousing concepts and ETL/ELT tools.

Mansioni

• Design, develop, and optimize scalable data pipelines for ingesting, processing, and transforming large volumes of data.
• Build and maintain data warehousing solutions and data lakes using big data technologies.
• Implement data quality frameworks and ensure accuracy, consistency, and reliability of data.
• Develop ETL/ELT processes to support BI and analytics requirements.
• Collaborate with data scientists, analysts, and stakeholders to understand data needs and deliver solutions.
• Monitor and troubleshoot data pipeline performance and data infrastructure issues.
• Stay updated with emerging big data technologies and best practices.

Conoscenze

Python

Scala

Java

SQL

ETL/ELT

Data Modeling

Formazione

Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or related field

Strumenti

Apache Spark

Hadoop

HDFS

Kafka

AWS EMR

S3

Redshift

Azure Data Lake

Databricks

GCP BigQuery

Descrizione del lavoro

Placements24 is seeking a Senior Data Engineer in South Africa with a strong background in big data and analytics. This remote-first role involves designing, building, and maintaining scalable data pipelines and data lakes to support advanced analytics and decision-making.

You will work with Spark, Hadoop, and cloud platforms to ensure data quality, security, and performance, collaborating with data scientists and business stakeholders in a distributed team environment.