Data Engineer: Cloud Pipelines & AI-Ready Data

Indsafri; South Africa; Hybrid; ZAR 600,000 - 900,000; Full time

Job summary

Indsafri is seeking a Data Engineer to design, build, and maintain robust data infrastructure. You will develop and optimize data pipelines, including ingestion, streaming, and API solutions, ensuring high-quality data for analytics and AI initiatives.

You will monitor pipelines, ensure security and reliability, and collaborate with Data Analysts to document and validate data assets. A strong focus on SLAs and enterprise architecture will shape daily work.

Qualifications

• Proven experience in data engineering, including data ingestion, transformation, and provisioning (ETL/ELT).
• Strong experience in building and maintaining data pipelines and streaming technologies.
• Solid understanding of data virtualization, SQL, and database management.
• Proficiency in monitoring and managing cloud infrastructure (compute and storage).
• Strong data profiling, quality assurance, and validation skills.
• Ability to provide first-line technical support and troubleshoot complex pipeline issues.
• Excellent collaboration skills with the ability to translate business needs into technical data solutions.

Responsibilities

• Manage day-to-day data pipeline operations, including data profiling, cleaning, configuration, and validation to support data squads and epics.
• Ensure the security, availability, and reliability of the data infrastructure.
• Maintain end-to-end data pipelines, spanning data virtualization, ingestion, and provisioning.
• Ensure monitoring systems are active and pipelines run successfully, with the ability to implement minor configuration changes.
• Develop virtual databases and generate precise data extracts to meet evolving business requirements.
• Partner with Data Analysts to perform data profiling, validation, and comprehensive documentation to support project epics.
• Provide first-line support for the Data Warehouse, ensuring strict adherence to Service Level Agreements (SLAs) regarding data availability and system reliability.
• Monitor and manage cloud-based compute and storage processes to ensure the seamless execution of cloud pipelines.
• Generate daily operational reports to track job success rates and ensure the data warehouse is maintained to enterprise standards.
• Collaborate with business stakeholders to gain domain knowledge, translate business requirements into technical solutions, and optimize business queries.

Skills

ETL/ELT

Data pipelines

Streaming tech

SQL

Cloud infrastructure

Data profiling

Data QA

Troubleshooting

Collaboration

Education

BSc Computer Science/IT

Cloud certifications (AWS/Azure/GCP)

Tools

SQL Server

Airflow

Kafka

Data virtualization tools

Job description

Indsafri is seeking a Data Engineer to design, build, and maintain robust data infrastructure. You will develop and optimize data pipelines, including ingestion, streaming, and API solutions, ensuring high-quality data for analytics and AI initiatives.

You will monitor pipelines, ensure security and reliability, and collaborate with Data Analysts to document and validate data assets. A strong focus on SLAs and enterprise architecture will shape daily work.