Data Engineer - Big Data & AI Platforms

Placements24; LegKraal Gate; Hybrid; ZAR 700,000 - 1,100,000; Full time

Benefits offered by this job

Health insurance

retirement savings

Performance bonuses

Job summary

Placements24 is seeking a Data Engineer in Mahikeng, North West to design and maintain scalable data pipelines supporting AI/ML initiatives. You will work with big data tech and cloud services to ingest, store, and process large datasets, enabling data-driven insights.

Collaboration with data scientists and engineers is essential, with a focus on quality, security, and performance. The ideal candidate has 4+ years in data engineering, strong SQL skills, and hands-on experience with Spark,

Qualifications

• Bachelor's or Master's degree in CS/Engineering/IS or related field.
• Minimum 4 years of data engineering experience.
• Strong SQL and relational DB experience.
• Hands-on experience with Spark, Hadoop, Kafka.
• Experience with cloud platforms (AWS, Azure, GCP) and data services.
• Proficiency in Python or Scala for data processing.
• Experience with data warehousing concepts and Kimball/Inmon methodologies.
• Excellent problem-solving and communication skills.
• Experience with Docker is a plus.

Responsibilities

• Design, build, and maintain scalable data pipelines using big data technologies and cloud services.
• Develop and optimize ETL/ELT processes for ingestion, transformation, and loading.
• Implement data warehousing solutions and data lakes for analytics and AI/ML workloads.
• Ensure data quality, integrity, and security across data platforms.
• Collaborate with data scientists, analysts, and engineers on data requirements.
• Monitor and troubleshoot data pipelines for performance and reliability.
• Document data architectures and governance practices.

Skills

SQL proficiency

Data modeling

Problem solving

Communication skills

Education

Bachelor's or Master's degree in Computer Science/Engineering/Information Systems

Tools

PostgreSQL

MySQL

Apache Spark

Hadoop

Kafka

Python

Scala

Job description

About the Role

Our client, a rapidly expanding data analytics firm, is seeking a talented Data Engineer to build and maintain robust data infrastructure supporting their advanced AI and machine learning initiatives. Located in Mahikeng, North West , this role is essential for ensuring the efficient collection, storage, processing, and accessibility of large datasets. You will work with cutting-edge big data technologies and cloud platforms to create scalable and reliable data pipelines that empower data scientists and analysts. The ideal candidate will possess strong skills in data architecture, ETL processes, and big data technologies, coupled with a passion for enabling data-driven insights. This is an excellent opportunity to contribute to a growing company in Mahikeng, North West , working on impactful data projects.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines using big data technologies and cloud services.
• Develop and optimize ETL/ELT processes for data ingestion, transformation, and loading.
• Implement data warehousing solutions and data lakes to support analytics and AI/ML workloads.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists, analysts, and other engineers to understand data requirements and provide solutions.
• Optimize data infrastructure for performance, cost-efficiency, and scalability.
• Monitor and troubleshoot data pipeline issues, ensuring data availability and accuracy.
• Stay up-to-date with emerging trends and technologies in big data and data engineering.
• Develop and maintain comprehensive documentation for data architectures and pipelines.
• Implement data governance best practices and policies.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related quantitative field.
• Minimum of 4 years of professional experience in data engineering or a similar role.
• Strong proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL).
• Hands-on experience with big data technologies such as Apache Spark, Hadoop, Kafka.
• Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, Redshift, BigQuery, Data Factory, EMR).
• Proficiency in programming languages like Python or Scala for data processing.
• Experience with data warehousing concepts and Kimball/Inmon methodologies.
• Familiarity with data modeling techniques and database design.
• Excellent problem-solving, analytical, and communication skills.
• Experience with containerization (Docker) is a plus.

Benefits

Our client offers a competitive salary and a comprehensive benefits package, including health insurance, retirement savings, and performance bonuses. We provide opportunities for continuous professional development, including access to training, certifications, and conferences. Enjoy a flexible work environment with hybrid working arrangements. This role is based in our modern offices in Mahikeng, North West . You will have the chance to work with a talented team on challenging data projects that drive business value and innovation. We foster a collaborative and inclusive culture that encourages growth and learning. Additional perks include team-building events and a vibrant work atmosphere. Join us in shaping the future of data analytics.