AI Data Engineer: Build Scalable Pipelines for ML

Placements24; Mtubatuba Local Municipality; Hybrid; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive salary

Medical, dental, and vision insurance

Generous annual leave

Job summary

Placements24 is seeking an experienced AI Data Engineer to join our Richards Bay team. You will design, build, and optimize scalable data pipelines for AI/ML workloads, ensuring data availability and quality for model training and deployment.

The role requires 5+ years in data engineering, strong SQL and Python/Java skills, and hands-on experience with Spark, Hadoop, Kafka, and Airflow, across AWS/Azure/GCP data services. Collaboration with data scientists and software teams is key.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related quantitative field.
• Minimum of 5 years of experience in data engineering, with a focus on AI/ML projects.
• Proficiency in SQL and programming languages such as Python or Java.
• Hands-on experience with big data technologies (e.g., Apache Spark, Hadoop, Kafka).
• Experience with cloud platforms and their data services (AWS, Azure, or GCP).
• Familiarity with data warehousing concepts and technologies.
• Knowledge of data modeling and database design principles.
• Experience with containerization technologies like Docker and Kubernetes is a plus.
• Strong understanding of data pipeline orchestration tools (e.g., Airflow).
• Excellent problem-solving and analytical skills.
• Good communication and teamwork abilities.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
• Develop and manage ETL/ELT processes to ingest, transform, and load data from various sources.
• Implement data quality checks and validation frameworks to ensure data accuracy and consistency.
• Optimize data storage and retrieval performance for large datasets.
• Collaborate with data scientists and ML engineers to understand their data requirements.
• Work with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP Cloud Storage, BigQuery).
• Implement and manage data governance and security policies.
• Develop data models and schemas to support analytical and machine learning needs.
• Monitor data pipeline performance and troubleshoot any issues that arise.
• Stay updated with emerging trends and technologies in data engineering and big data.

Skills

SQL

Python

Java

Apache Spark

Hadoop

Kafka

Airflow

Cloud Platforms

Data Warehousing

Data Modeling

Docker

Kubernetes

ETL/ELT

Problem Solving

Communication

Education

Bachelor's degree in CS/Engineering/IT

Tools

Apache Spark

Hadoop

Kafka

Airflow

AWS

Azure

GCP

Docker

Kubernetes

Redshift

BigQuery

Job description

Placements24 is seeking an experienced AI Data Engineer to join our Richards Bay team. You will design, build, and optimize scalable data pipelines for AI/ML workloads, ensuring data availability and quality for model training and deployment.

The role requires 5+ years in data engineering, strong SQL and Python/Java skills, and hands-on experience with Spark, Hadoop, Kafka, and Airflow, across AWS/Azure/GCP data services. Collaboration with data scientists and software teams is key.