AI Data Engineer: Build Scalable Data Pipelines

Indsafri; Johannesburg; On-site; ZAR 600,000 - 1,000,000; Full time

Job summary

Indsafri seeks a highly skilled Data Engineer focused on AI to design, build, and maintain data pipelines and infrastructure that support AI and ML initiatives. You will work with data scientists and ML engineers to ensure data quality, accessibility, and efficiency for model development and deployment.

You will implement governance, monitoring, and cost-efficient data processing, staying current with AI/ML data technologies and best practices.

Qualifications

• Bachelor's or Master's degree in a related field.
• Proven experience as a Data Engineer.
• Strong SQL proficiency with relational and NoSQL databases.
• Expertise in Python, Java, or Scala.
• Experience with Spark, Hadoop, Kafka, or Flink.
• Hands-on cloud experience (AWS, Azure, GCP).
• Familiarity with data warehousing and ETL/ELT processes.
• Understanding of ML concepts and data requirements for AI models.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.

Responsibilities

• Design, develop, and optimize scalable data pipelines for AI/ML applications.
• Build and maintain data infrastructure (data warehouses, data lakes, real-time systems).
• Implement data governance for accuracy, consistency, and security.
• Collaborate with data scientists and ML engineers on data needs.
• Develop monitoring and alerting for data pipelines and infrastructure.
• Optimize data processing for performance and cost efficiency.
• Stay updated with trends in data engineering, AI, and ML.
• Troubleshoot data-related issues in production.

Skills

SQL proficiency

Analytical thinking

Problem solving

Communication

Team collaboration

Education

Bachelor's or Master's in CS/Engineering/Math

Master's degree preferred

Tools

Python

Java

Scala

Spark

Hadoop

Kafka

Flink

AWS

Azure

GCP

ETL/ELT

Data warehousing

Docker

Kubernetes

Job description

Indsafri seeks a highly skilled Data Engineer focused on AI to design, build, and maintain data pipelines and infrastructure that support AI and ML initiatives. You will work with data scientists and ML engineers to ensure data quality, accessibility, and efficiency for model development and deployment.

You will implement governance, monitoring, and cost-efficient data processing, staying current with AI/ML data technologies and best practices.