AI/ML Data Engineer: Scalable Pipelines & Data Lake

Competent Candidates; Gauteng; Hybrid; ZAR 720,000 - 980,000; Full time

Benefits offered by this job

Competitive salary

Hybrid work model

Growth opportunities

Job summary

Competent Candidates in Soweto is seeking an AI/ML Data Engineer to build and maintain data pipelines and infrastructure supporting AI initiatives. You will collect, clean, transform, and manage large datasets, ensuring they are optimized for model training and analysis in a hybrid work setting.

You will collaborate with data scientists and AI researchers to provide high-quality data foundations for developing and deploying sophisticated AI solutions, unlocking data-driven insights.

Qualifications

• 3+ years of experience in data engineering, with a focus on AI/ML projects.
• Proficiency in SQL and programming languages such as Python or Scala.
• Experience with big data platforms like Spark, Hadoop, or similar.
• Familiarity with cloud data services (AWS, Azure, GCP) and data warehousing concepts.
• Strong understanding of data governance, data quality, and data security principles.

Responsibilities

• Design, build, and maintain scalable data pipelines for AI and machine learning workloads.
• Develop ETL processes to ingest, clean, transform, and aggregate data from various sources.
• Ensure data quality, integrity, and accessibility for data scientists and analysts.
• Implement data warehousing and data lake solutions to support big data analytics.
• Optimize data infrastructure for performance, scalability, and cost-effectiveness.
• Collaborate with data scientists to understand data requirements and prepare data for model training and evaluation.

Job description

Competent Candidates in Soweto is seeking an AI/ML Data Engineer to build and maintain data pipelines and infrastructure supporting AI initiatives. You will collect, clean, transform, and manage large datasets, ensuring they are optimized for model training and analysis in a hybrid work setting.

You will collaborate with data scientists and AI researchers to provide high-quality data foundations for developing and deploying sophisticated AI solutions, unlocking data-driven insights.