Data Engineer — AI Platform Pipelines (On-Site)

Placements24; Centurion; On-site; ZAR 650,000 - 900,000; Full time

Benefits offered by this job

Competitive salary

On-site facilities

Medical, dental, and vision insurance

Professional development and certs

Collaborative team environment

Job summary

Placements24 is seeking a meticulous Data Engineer to design, build, and optimize scalable data pipelines for AI and ML initiatives in Centurion, on-site. You will ensure data availability, quality, and accessibility for ML models while collaborating with data scientists and ML engineers to craft robust data solutions.

The role emphasizes data governance, performance optimization, and cost efficiency within an innovative, collaborative team environment.

Qualifications

• Bachelor's or Master's degree in CS/Engineering or a related field.
• 3+ years of data engineering experience building large-scale pipelines.
• Proficiency in SQL and programming languages like Python or Scala.
• Experience with Spark, Hadoop, Kafka, and distributed storage systems.

Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Develop processes and systems to monitor data quality, ensure consistency, and manage data flow.
• Build and manage ETL/ELT processes for large, diverse datasets.
• Optimize data infrastructure for performance, reliability, and cost-effectiveness.
• Collaborate with data scientists and ML engineers to understand data requirements and provide solutions.
• Implement data security and governance best practices.

Skills

SQL

Python

Scala

Education

Bachelor's or Master's degree in CS/Engineering or related field

Tools

Spark

Hadoop

Kafka

Distributed storage

AWS

Azure

GCP

Job description

Placements24 is seeking a meticulous Data Engineer to design, build, and optimize scalable data pipelines for AI and ML initiatives in Centurion, on-site. You will ensure data availability, quality, and accessibility for ML models while collaborating with data scientists and ML engineers to craft robust data solutions.

The role emphasizes data governance, performance optimization, and cost efficiency within an innovative, collaborative team environment.