Hybrid Senior Data Engineer: Cloud Pipelines for AI

Placements24; Vrystaat; Hybrid; ZAR 900,000 - 1,500,000; Full time

Benefits offered by this job

Competitive salary

Medical aid and retirement benefits

Training programs and conferences

Hybrid work arrangement

Job summary

Placements24 is seeking a Senior Data Engineer to design and maintain scalable data pipelines and cloud data infrastructure for AI/ML workloads in Upington, Northern Cape. The role focuses on data availability, quality, and security, collaborating with data scientists and engineers in a hybrid work model.

You will implement ETL/ELT processes, develop data warehousing solutions, and optimize data models for performance across AWS, Azure, and GCP environments, using Spark, Hadoop, and SQL.

Qualifications

• Bachelor's or Master's in CS/IT/Engineering or related quantitative field.
• Minimum 5 years' experience in data engineering and pipelines.
• Proven cloud data services experience across AWS/Azure/GCP.
• Strong SQL, and big data tech like Spark, Hadoop, Kafka.
• Python for data manipulation and scripting; ETL/ELT design.
• Understanding of data governance, quality, and security.

Responsibilities

• Design, build, and maintain scalable ETL data pipelines for large datasets.
• Develop and manage cloud-based data infrastructure for AI/ML workloads.
• Implement data warehousing and optimize models for performance.
• Collaborate with data scientists and AI engineers to meet data needs.
• Ensure data quality, integrity, and security across platforms.
• Automate data processes and monitor pipelines with alerts.
• Optimize data retrieval for complex analytics and model training.
• Stay current with emerging data engineering technologies.
• Troubleshoot data-related issues to minimize AI/ML disruption.
• Contribute to overall data strategy and architecture.

Skills

SQL

Python

Spark

Hadoop

Kafka

Docker

Data Warehousing

Cloud Platforms

Education

Bachelor's or Master's degree in CS/IT/Engineering

Master's degree preferred

Tools

AWS

Azure

GCP

Redshift

BigQuery

Glue

Synapse

Data Factory

Dataflow

Docker

Job description

Placements24 is seeking a Senior Data Engineer to design and maintain scalable data pipelines and cloud data infrastructure for AI/ML workloads in Upington, Northern Cape. The role focuses on data availability, quality, and security, collaborating with data scientists and engineers in a hybrid work model.

You will implement ETL/ELT processes, develop data warehousing solutions, and optimize data models for performance across AWS, Azure, and GCP environments, using Spark, Hadoop, and SQL.