Data Engineer - AI Platform

Placements24; Centurion; On-site; ZAR 650,000 - 900,000; Full time

Benefits offered by this job

Competitive salary

On-site facilities

Medical, dental, and vision insurance

Professional development and certs

Collaborative team environment

Job summary

Placements24 is seeking a meticulous Data Engineer to design, build, and optimize scalable data pipelines for AI and ML initiatives in Centurion, on-site. You will ensure data availability, quality, and accessibility for ML models while collaborating with data scientists and ML engineers to craft robust data solutions.

The role emphasizes data governance, performance optimization, and cost efficiency within an innovative, collaborative team environment.

Qualifications

• Bachelor's or Master's degree in CS/Engineering or a related field.
• 3+ years of data engineering experience building large-scale pipelines.
• Proficiency in SQL and programming languages like Python or Scala.
• Experience with Spark, Hadoop, Kafka, and distributed storage systems.

Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Develop processes and systems to monitor data quality, ensure consistency, and manage data flow.
• Build and manage ETL/ELT processes for large, diverse datasets.
• Optimize data infrastructure for performance, reliability, and cost-effectiveness.
• Collaborate with data scientists and ML engineers to understand data requirements and provide solutions.
• Implement data security and governance best practices.

Skills

SQL

Python

Scala

Education

Bachelor's or Master's degree in CS/Engineering or related field

Tools

Spark

Hadoop

Kafka

Distributed storage

AWS

Azure

GCP

Job description

About the Role

Our client is seeking a meticulous and experienced Data Engineer to build and optimize robust data pipelines for their AI & Emerging Technologies initiatives in Centurion . In this critical role, you will be responsible for ensuring the availability, quality, and accessibility of data required for machine learning models and AI-driven applications. You will work closely with data scientists and ML engineers to architect scalable data solutions, fostering a data-centric culture. This on-site position provides a unique opportunity to work with a dedicated team in a collaborative and innovative environment.

Key Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Develop processes and systems to monitor data quality, ensure consistency, and manage data flow.
• Build and manage ETL/ELT processes for large, diverse datasets.
• Optimize data infrastructure for performance, reliability, and cost-effectiveness.
• Collaborate with data scientists and ML engineers to understand their data requirements and provide solutions.
• Implement data security and governance best practices.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering, with a focus on building large-scale data pipelines.
• Proficiency in SQL and programming languages like Python or Scala.
• Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed storage systems.
• Hands-on experience with cloud data services (AWS, Azure, GCP).
• Strong understanding of data warehousing concepts and database design.

Benefits

• Competitive annual salary and performance incentives.
• On-site work environment with access to advanced facilities.
• Comprehensive medical, dental, and vision insurance plans.
• Opportunities for professional development and certifications.
• Collaborative team atmosphere focused on innovation and growth.