AI Data Platform Engineer – Scale and Optimize Pipelines

Placements24; Pretoria; On-site; ZAR 900,000 - 1,200,000; Full time

Benefits offered by this job

Competitive compensation

Medical, dental & vision insurance

Pension plan

Career growth in Pretoria

Collaborative environment

Job summary

Placements24 in Pretoria seeks a Data Engineer to build and maintain robust data pipelines for their AI platform. You will design, construct, install, test, and maintain scalable data management systems, ensuring high-quality data for machine learning models and AI applications.

Join a team leveraging big data technologies on cloud platforms and collaborate with AI/ML engineers to address diverse data requirements, while upholding data security and regulatory standards.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of data engineering experience building data pipelines for AI/ML.
• Strong proficiency in SQL and Python or Java.
• Experience with Spark, Hadoop, Kafka and distributed databases.
• Hands-on experience with cloud data services (AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP BigQuery).
• Familiarity with data warehousing concepts and data modeling best practices.

Responsibilities

• Design, build, and maintain scalable and reliable ETL/ELT pipelines for ingesting and processing diverse data sources.
• Develop and optimize data models and database structures to support AI and machine learning workloads.
• Implement data quality checks and monitoring systems to ensure data integrity and accuracy.
• Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary data solutions.
• Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
• Ensure data security and compliance with relevant regulations.

Skills

SQL

Python

Java

ETL/ELT pipelines

Data modeling

Data warehousing

Data quality checks

Cloud platforms

Big data technologies

Collaboration with AI/ML teams

Education

Bachelor's degree in Computer Science, Engineering, or a related field

Tools

Spark

Hadoop

Kafka

Redshift

Glue

Azure Data Lake

Synapse

BigQuery

S3

Job description

Placements24 in Pretoria seeks a Data Engineer to build and maintain robust data pipelines for their AI platform. You will design, construct, install, test, and maintain scalable data management systems, ensuring high-quality data for machine learning models and AI applications.

Join a team leveraging big data technologies on cloud platforms and collaborate with AI/ML engineers to address diverse data requirements, while upholding data security and regulatory standards.