Data Engineer - AI Platforms

Placements24; Rustenburg; Hybrid; ZAR 650,000 - 900,000; Full time

Benefits offered by this job

Competitive salary

Health insurance

Pension plan

Professional development

Rustenburg-based team environment

Job summary

Placements24 in Rustenburg is seeking a skilled Data Engineer to design, build, and maintain robust data infrastructure for AI initiatives. You''ll collaborate with data scientists and ML engineers to ensure reliable data for models and analytics, focusing on scalable pipelines and efficient storage.

The role requires a 3+ year background in data engineering, SQL and Python, plus hands-on with Airflow and big data frameworks in cloud environments.

Qualifications

• Bachelor's degree in CS, Engineering, or related field.
• 3+ years of data engineering experience with big data tech.
• Proficient in SQL and Python.
• Experience with Airflow and distributed processing frameworks.
• Familiar with cloud data services (AWS, Azure).

Responsibilities

• Design and maintain scalable data pipelines for AI/ML workloads.
• Develop data warehousing and data lake architectures.
• Implement data quality checks and ensure data integrity.
• Optimize processing performance and cost efficiency.
• Collaborate with data scientists and ML engineers to meet data needs.

Skills

SQL

Python

Airflow

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Airflow

Spark

Hadoop

AWS S3

Redshift

Glue

Azure Data Factory

Synapse

Job description

About the Role

Our client is seeking a skilled Data Engineer to build and maintain robust data infrastructure for their AI initiatives in Rustenburg. This role is crucial for ensuring the availability, quality, and accessibility of data that powers their advanced machine learning models and analytics platforms. You will work closely with data scientists and ML engineers to design efficient data pipelines and storage solutions. This position offers a chance to make a significant impact on the company's data strategy and contribute to the scaling of its AI capabilities.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
• Develop and manage data warehousing solutions and data lake architectures.
• Implement data quality checks and ensure data integrity and consistency.
• Optimize data processing performance and cost efficiency.
• Collaborate with cross-functional teams to understand data needs and provide solutions.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering, with a focus on big data technologies.
• Proficiency in SQL, Python, and data pipeline orchestration tools (e.g., Airflow).
• Experience with distributed data processing frameworks like Spark or Hadoop.
• Familiarity with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Factory, Synapse).

Benefits

• Competitive salary and annual bonus opportunities.
• Comprehensive health, disability, and life insurance.
• Pension fund contribution and retirement planning support.
• Professional development programs and training opportunities.
• A collaborative and innovative work environment in Rustenburg.