Senior Data Engineer - AI Platforms

Placements24; Vereeniging; Hybrid; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive salary

Health benefits

Professional development opportunities

Job summary

Placements24 seeks a Senior Data Engineer to build and maintain scalable data pipelines and infrastructure for AI platforms. This fully remote role can be performed from anywhere, including Vereeniging, Gauteng, SA, and will collaborate with data scientists to support ML initiatives.

The ideal candidate has experience with Spark, Hadoop, Kafka, and cloud data services, plus strong SQL and programming skills. Excellent collaboration in a remote team is essential.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, IT, or related field.
• Proven experience building large-scale data pipelines.
• Experience with cloud data platforms and data warehousing concepts.

Responsibilities

• Design, build, test, and maintain scalable data pipelines and infrastructure.
• Develop and optimize ETL/ELT processes for ML and analytics.
• Manage data lakes/warehouses ensuring quality and accessibility.
• Collaborate with data scientists and AI engineers on data needs.
• Implement data governance and security practices.
• Monitor pipelines, troubleshoot, and improve performance.
• Automate data processes and CI/CD tooling; evaluate new tech.

Skills

SQL

Python

Scala

Java

Data Pipelines

Data Modeling

Distributed Systems

CI/CD

Education

Bachelor's or Master's degree in CS/Engineering/IT

Tools

Apache Spark

Hadoop

Kafka

S3

Redshift

BigQuery

Databricks

Job description

About the Role

Our client, a prominent technology firm dedicated to advancing AI and data science, is seeking a Senior Data Engineer to bolster their team. This role is based in Vereeniging, Gauteng, ZA , but is offered as a fully remote position, allowing top talent to contribute from anywhere. You will be instrumental in building and maintaining robust, scalable data pipelines and infrastructure that support our cutting-edge AI platforms and machine learning initiatives. This is an excellent opportunity to work with large-scale datasets and contribute to critical data-driven products.

Key Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Develop and optimize ETL/ELT processes for ingesting, transforming, and preparing data for machine learning models and analytics.
• Build and manage data lakes and data warehouses, ensuring data quality, reliability, and accessibility.
• Collaborate with data scientists and AI engineers to understand their data requirements and provide solutions.
• Implement data governance and security best practices within the data infrastructure.
• Monitor data pipeline performance, troubleshoot issues, and implement solutions to ensure efficiency and uptime.
• Automate data processes and infrastructure management using scripting and CI/CD tools.
• Evaluate and recommend new data technologies and tools to enhance the data platform.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
• Proven experience as a Data Engineer, with a strong focus on building and managing large-scale data pipelines.
• Proficiency in SQL and programming languages such as Python, Scala, or Java.
• Hands-on experience with big data technologies like Apache Spark, Hadoop, Kafka, and distributed systems.
• Experience with cloud data platforms (AWS, Azure, GCP) and their data services (e.g., S3, Redshift, BigQuery, Databricks).
• Familiarity with data warehousing concepts and database technologies (relational and NoSQL).
• Understanding of machine learning workflows and data requirements for AI models.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills, essential for a remote team.

Benefits

• Our client offers a competitive salary, performance bonuses, and a comprehensive benefits package, including health, retirement, and professional development opportunities.
• Experience the freedom and flexibility of a fully remote role while contributing to impactful AI projects.
• Join a forward-thinking company that values innovation, collaboration, and employee growth.
• This role in Vereeniging, Gauteng, ZA offers a unique chance to shape the future of data infrastructure for AI.