AI Data Engineer

Placements24; Mtubatuba Local Municipality; Hybrid; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive salary

Medical, dental, and vision insurance

Generous annual leave

Job summary

Placements24 is seeking an experienced AI Data Engineer to join our Richards Bay team. You will design, build, and optimize scalable data pipelines for AI/ML workloads, ensuring data availability and quality for model training and deployment.

The role requires 5+ years in data engineering, strong SQL and Python/Java skills, and hands-on experience with Spark, Hadoop, Kafka, and Airflow, across AWS/Azure/GCP data services. Collaboration with data scientists and software teams is key.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related quantitative field.
• Minimum of 5 years of experience in data engineering, with a focus on AI/ML projects.
• Proficiency in SQL and programming languages such as Python or Java.
• Hands-on experience with big data technologies (e.g., Apache Spark, Hadoop, Kafka).
• Experience with cloud platforms and their data services (AWS, Azure, or GCP).
• Familiarity with data warehousing concepts and technologies.
• Knowledge of data modeling and database design principles.
• Experience with containerization technologies like Docker and Kubernetes is a plus.
• Strong understanding of data pipeline orchestration tools (e.g., Airflow).
• Excellent problem-solving and analytical skills.
• Good communication and teamwork abilities.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
• Develop and manage ETL/ELT processes to ingest, transform, and load data from various sources.
• Implement data quality checks and validation frameworks to ensure data accuracy and consistency.
• Optimize data storage and retrieval performance for large datasets.
• Collaborate with data scientists and ML engineers to understand their data requirements.
• Work with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP Cloud Storage, BigQuery).
• Implement and manage data governance and security policies.
• Develop data models and schemas to support analytical and machine learning needs.
• Monitor data pipeline performance and troubleshoot any issues that arise.
• Stay updated with emerging trends and technologies in data engineering and big data.

Skills

SQL

Python

Java

Apache Spark

Hadoop

Kafka

Airflow

Cloud Platforms

Data Warehousing

Data Modeling

Docker

Kubernetes

ETL/ELT

Problem Solving

Communication

Education

Bachelor's degree in CS/Engineering/IT

Tools

Apache Spark

Hadoop

Kafka

Airflow

AWS

Azure

GCP

Docker

Kubernetes

Redshift

BigQuery

Job description

About the Role

Our client, a dynamic company operating at the cutting edge of AI and Emerging Technologies, is looking for a skilled AI Data Engineer to join their team in Richards Bay, KwaZulu-Natal. This role is central to building and maintaining the robust data infrastructure required to support advanced AI and machine learning initiatives. You will be responsible for designing, developing, and optimizing data pipelines, ensuring the availability, quality, and accessibility of data for AI model training and deployment. The ideal candidate possesses strong expertise in data warehousing, ETL processes, big data technologies, and cloud-based data platforms. You will collaborate closely with data scientists, ML engineers, and software developers to enable efficient data management and drive data-driven decision-making across the organization. This position offers a fantastic opportunity to contribute to impactful AI projects in a growing sector and work within a collaborative team environment.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
• Develop and manage ETL/ELT processes to ingest, transform, and load data from various sources.
• Implement data quality checks and validation frameworks to ensure data accuracy and consistency.
• Optimize data storage and retrieval performance for large datasets.
• Collaborate with data scientists and ML engineers to understand their data requirements.
• Work with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP Cloud Storage, BigQuery).
• Implement and manage data governance and security policies.
• Develop data models and schemas to support analytical and machine learning needs.
• Monitor data pipeline performance and troubleshoot any issues that arise.
• Stay updated with emerging trends and technologies in data engineering and big data.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field.
• Minimum of 5 years of experience in data engineering, with a focus on supporting AI/ML projects.
• Proficiency in SQL and programming languages such as Python or Java.
• Hands-on experience with big data technologies (e.g., Apache Spark, Hadoop, Kafka).
• Experience with cloud platforms and their data services (AWS, Azure, or GCP).
• Familiarity with data warehousing concepts and technologies.
• Knowledge of data modeling and database design principles.
• Experience with containerization technologies like Docker and Kubernetes is a plus.
• Strong understanding of data pipeline orchestration tools (e.g., Airflow).
• Excellent problem-solving and analytical skills.
• Good communication and teamwork abilities.

Benefits

• Competitive salary and potential for bonuses.
• Comprehensive medical, dental, and vision insurance.
• Generous annual leave allowance.
• Opportunities for professional development and training.
• Exposure to cutting-edge AI and big data technologies.
• Collaborative and innovative work environment.
• Retirement savings plan.
• Performance recognition programs.