Senior Data Engineer - Big Data Technologies

Placements24; Randburg; Hybrid; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive salary and annual bonus

Medical, dental, and vision benefits

Retirement savings plan with employer

Training and professional development

Hybrid work environment

Job summary

Placements24 in Randburg is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support analytics, BI, and machine learning initiatives.

You will work with large datasets and modern big data technologies (Spark, Hadoop, Kafka) on cloud platforms, collaborating with data scientists and business stakeholders to translate raw data into reliable insights and business value.

Qualifications

• Degree in a quantitative field as listed.
• 5+ years in data engineering or related roles.
• Strong SQL and data warehousing experience.
• Hands-on with Spark, Hadoop, Kafka or similar.
• Proficient in Python or Scala and cloud platforms.

Responsibilities

• Design, construct, install, test, and maintain scalable data pipelines.
• Develop ETL/ELT processes for data ingestion and transformation.
• Optimize data processing for performance and cost.
• Collaborate with data scientists, analysts, and stakeholders.
• Ensure data quality, integrity, and security.
• Build data models to support reporting and ML applications.

Skills

SQL proficiency

Python/Scala programming

Data modeling

Education

Bachelor's or Master's in CS/Engineering/Statistics

Tools

Apache Spark

Hadoop

Kafka

NoSQL databases

Job description

About the Role

Our client is seeking a skilled Senior Data Engineer to join their analytics division in Randburg . This role is vital in designing, building, and maintaining scalable data pipelines and infrastructure that support advanced analytics and business intelligence initiatives. You will work with large datasets and cutting-edge big data technologies to transform raw data into actionable insights. This is an excellent opportunity for an experienced data professional to contribute to data-driven decision-making within a forward-thinking organization that values innovation and professional development.

Key Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Develop and implement ETL/ELT processes to ingest, transform, and load data from various sources into data warehouses and data lakes.
• Optimize data retrieval and processing for performance and cost-efficiency using big data technologies like Spark and Hadoop.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
• Ensure data quality, integrity, and security across all data systems.
• Build and maintain data models that support reporting, analytics, and machine learning applications.

Requirements

• Master's or Bachelor's degree in Computer Science, Engineering, Statistics, or a related quantitative field.
• 5+ years of experience in data engineering or a related role.
• Proficiency in SQL and experience with relational and NoSQL databases.
• Hands-on experience with big data technologies such as Apache Spark , Hadoop, Kafka, or similar.
• Strong programming skills in languages like Python or Scala.
• Experience with cloud platforms (AWS, Azure, GCP) and data warehousing solutions.

Benefits

• Competitive salary and potential for annual bonuses.
• Comprehensive medical, dental, and vision benefits.
• Retirement savings plan with employer matching.
• Generous opportunities for training and professional development.
• A supportive hybrid work environment that promotes work-life balance.