AI/ML Data Engineer - Big Data Platforms

Placements24; LegKraal Gate; Vor Ort; ZAR 600.000 - 900.000; Vollzeit

Benefits dieser Stelle

Health insurance

Retirement plans

Growth opportunities

Zusammenfassung

Placements24 is seeking an AI/ML Data Engineer to design, build, and maintain scalable data pipelines to support AI initiatives in Mahikeng, North West. You will work with data scientists and ML engineers, optimize ETL/ELT processes, ensure data quality and governance, and apply big data tools like Spark, Hadoop, and Kafka on cloud platforms (AWS/Azure/GCP).

Ideal candidates have 3+ years in data engineering, strong SQL, programming in Python/Scala/Java, and solid data modeling experience.

Qualifikationen

• Bachelor's or Master's in CS/Engineering or related field.
• 3+ years in data engineering for AI/ML projects.
• Proficiency in SQL and relational/NoSQL databases.
• Experience with Spark, Hadoop, Kafka, or similar.
• Cloud data platforms AWS/Azure/GCP and services like S3, Redshift, BigQuery.

Aufgaben

• Design, build, and maintain scalable data pipelines for AI/ML apps.
• Develop and manage ETL/ELT processes for ingestion, transformation, loading.
• Optimize storage and retrieval for large datasets using big data tech.
• Ensure data quality, integrity, and reliability across systems.
• Collaborate with AI/ML engineers and data scientists on data needs.
• Implement data governance and security best practices.
• Monitor data infrastructure performance and troubleshoot issues.
• Stay updated with trends in big data and data engineering.
• Contribute to architecture and design of the data platform.
• Document data processes, schemas, and infrastructure.

Kenntnisse

SQL

Python

Scala

Java

Data modeling

Big data

Teamwork

Communication

Ausbildung

Bachelor's degree

Master's degree

Tools

Spark

Hadoop

Kafka

AWS

Azure

GCP

S3

Redshift

BigQuery

NoSQL

Jobbeschreibung

About the Role

Our client, a forward-thinking company dedicated to leveraging data for transformative solutions, is seeking a skilled AI/ML Data Engineer to join their growing team in Mahikeng, North West . This role is integral to building and maintaining the robust data infrastructure required to support our advanced AI and machine learning initiatives. You will be responsible for designing, developing, and optimizing data pipelines, ensuring the availability, quality, and accessibility of data for our data scientists and ML engineers. The ideal candidate will possess a strong background in data engineering, big data technologies, and a solid understanding of the data needs of AI/ML workloads. This is an excellent opportunity to work with cutting-edge technologies and contribute to impactful projects from our Mahikeng, North West office.

Key Responsibilities

• Design, build, and maintain scalable and efficient data pipelines for AI/ML applications.
• Develop and manage ETL/ELT processes for data ingestion, transformation, and loading.
• Optimize data storage and retrieval for large datasets using big data technologies.
• Ensure data quality, integrity, and reliability across all data systems.
• Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary support.
• Implement data governance and security best practices.
• Monitor data infrastructure performance and troubleshoot issues as they arise.
• Stay up-to-date with emerging trends and technologies in big data and data engineering.
• Contribute to the architecture and design of our data platform.
• Document data processes, schemas, and infrastructure.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 3+ years of experience in data engineering, with a focus on supporting AI/ML projects.
• Proficiency in SQL and experience with relational and NoSQL databases.
• Strong experience with big data technologies such as Spark, Hadoop, Kafka, or similar.
• Experience with cloud data platforms (AWS, Azure, GCP) and their associated services (e.g., S3, Redshift, BigQuery, ADLS).
• Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java).
• Understanding of data warehousing concepts and data modeling techniques.
• Familiarity with MLOps principles and data infrastructure for ML is a plus.
• Excellent problem-solving and analytical skills.
• Strong teamwork and communication abilities for our Mahikeng, North West office.

Benefits

• Our client offers a competitive salary and benefits package, including health insurance and retirement plans.
• Employees enjoy a dynamic work environment with opportunities for professional growth and development.
• We provide access to advanced tools and technologies, and foster a culture of collaboration and innovation in Mahikeng, North West .
• Join our team and play a crucial role in building the data foundation for cutting-edge AI solutions.