Data Engineer - AI Platform

Placements24; Pretoria; On-site; ZAR 900,000 - 1,200,000; Full time

Benefits offered by this job

Competitive compensation

Medical, dental & vision insurance

Pension plan

Career growth in Pretoria

Collaborative environment

Job summary

Placements24 in Pretoria seeks a Data Engineer to build and maintain robust data pipelines for their AI platform. You will design, construct, install, test, and maintain scalable data management systems, ensuring high-quality data for machine learning models and AI applications.

Join a team leveraging big data technologies on cloud platforms and collaborate with AI/ML engineers to address diverse data requirements, while upholding data security and regulatory standards.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of data engineering experience building data pipelines for AI/ML.
• Strong proficiency in SQL and Python or Java.
• Experience with Spark, Hadoop, Kafka and distributed databases.
• Hands-on experience with cloud data services (AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP BigQuery).
• Familiarity with data warehousing concepts and data modeling best practices.

Responsibilities

• Design, build, and maintain scalable and reliable ETL/ELT pipelines for ingesting and processing diverse data sources.
• Develop and optimize data models and database structures to support AI and machine learning workloads.
• Implement data quality checks and monitoring systems to ensure data integrity and accuracy.
• Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary data solutions.
• Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
• Ensure data security and compliance with relevant regulations.

Skills

SQL

Python

Java

ETL/ELT pipelines

Data modeling

Data warehousing

Data quality checks

Cloud platforms

Big data technologies

Collaboration with AI/ML teams

Education

Bachelor's degree in Computer Science, Engineering, or a related field

Tools

Spark

Hadoop

Kafka

Redshift

Glue

Azure Data Lake

Synapse

BigQuery

S3

Job description

About the Role

Our client is looking for a skilled Data Engineer to build and maintain robust data pipelines for their advanced AI platform in Pretoria . You will be responsible for designing, constructing, installing, testing, and maintaining highly scalable data management systems. Your work will be crucial in ensuring that high-quality data is available for machine learning models and AI applications. This role offers a fantastic opportunity to work with large datasets, modern data technologies, and contribute significantly to the company's data-driven innovation strategy within the bustling capital city of Gauteng .

Key Responsibilities

• Design, build, and maintain scalable and reliable ETL/ELT pipelines for ingesting and processing diverse data sources.
• Develop and optimize data models and database structures to support AI and machine learning workloads.
• Implement data quality checks and monitoring systems to ensure data integrity and accuracy.
• Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary data solutions.
• Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
• Ensure data security and compliance with relevant regulations.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering, with a focus on building data pipelines for AI/ML applications.
• Strong proficiency in SQL and programming languages like Python or Java.
• Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed databases.
• Hands-on experience with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP BigQuery).
• Familiarity with data warehousing concepts and data modeling best practices.

Benefits

• Competitive salary and performance incentives.
• Comprehensive medical, dental, and vision insurance.
• Pension fund and other retirement benefits.
• Opportunities for career advancement and skill development in Pretoria .
• A collaborative work environment focused on innovation and data excellence.