Data Engineer - AI Platforms

Placements24; Randburg; Hybrid; ZAR 600,000 - 900,000; Full time

Benefits offered by this job

Competitive salary

Professional development opportunities

Hybrid work arrangement

Challenging AI projects

Collaborative engineering team

Job summary

Placements24 seeks a skilled Data Engineer to join its AI Platforms team in Randburg, focusing on building robust data pipelines and scalable data infrastructure for advanced AI initiatives. You will design ETL processes, ensure data quality, and enable data access for model training and analysis.

The role requires 3+ years in data engineering, strong SQL and Python/Scala skills, and experience with Spark/Hadoop and cloud platforms. A hybrid work setup in Randburg is offered.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering or a similar role.
• Proficiency in SQL and programming languages such as Python or Scala.
• Experience with big data technologies like Spark, Hadoop, and distributed databases.
• Familiarity with cloud data platforms (AWS, Azure, GCP) and services.
• Understanding of data modeling and database design principles.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML applications.
• Develop and optimize ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and accessibility for data scientists and analysts.
• Implement data warehousing solutions and data lake architectures.
• Collaborate with AI/ML engineers to understand their data needs and provide efficient solutions.
• Monitor and troubleshoot data infrastructure performance and resolve issues promptly.

Skills

SQL

Python/Scala

Data pipelines

Data modeling

Cloud platforms

Education

Bachelor's degree in Computer Science or Engineering

Tools

Spark

Hadoop

Distributed databases

Job description

About the Role

Our client is seeking a skilled Data Engineer to join their expanding AI Platforms team in Randburg . This role is integral to building and maintaining robust data infrastructure that powers our advanced AI and machine learning initiatives. You will be responsible for designing, developing, and optimizing data pipelines, ensuring data quality, and making large datasets accessible for analysis and model training. The ideal candidate has a strong background in data engineering principles and a passion for enabling data-driven innovation within a collaborative, fast-paced technology environment.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML applications.
• Develop and optimize ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and accessibility for data scientists and analysts.
• Implement data warehousing solutions and data lake architectures.
• Collaborate with AI/ML engineers to understand their data needs and provide efficient solutions.
• Monitor and troubleshoot data infrastructure performance and resolve issues promptly.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering or a similar role.
• Proficiency in SQL and programming languages such as Python or Scala.
• Experience with big data technologies like Spark, Hadoop, and distributed databases.
• Familiarity with cloud data platforms (AWS, Azure, GCP) and services.
• Understanding of data modeling and database design principles.

Benefits

• Competitive salary and comprehensive benefits package.
• Opportunities for professional development and training in Randburg .
• A hybrid work arrangement offering flexibility.
• Work on challenging projects that enable cutting-edge AI advancements.
• Join a collaborative and innovative engineering team.