Data Engineering Lead for AI & GenAI Platforms

DVT; Cape Town; On-site; ZAR 1,400,000 - 2,200,000; Full time

Job summary

DVT is seeking an experienced Data Engineering Lead to design, build and govern data and architecture for AI initiatives. You will own Azure- and AWS-based environments, develop governed data pipelines and collaborate with data scientists and engineers to productionise AI capabilities.

You will lead the data engineering team, ensure data quality and governance, and drive cost optimisation and incident management while aligning with security standards.

Qualifications

• 7+ years of data engineering, architecture and cloud platform experience.
• predecessor
• Needs strong cloud alignment and governance understanding.
• Azure and/or AWS certifications are a plus.
• Experience with Azure data & AI services and AWS/Azure data integration.
• Proven data pipelines: batch, streaming, API-based.
• Experience enabling Data Science/AI with MLOps capabilities.
• Strong Python/SQL, data modelling and transformations.
• Experience with Git-based CI/CD and IaC.
• Knowledge of data governance and enterprise architecture coordination.

Responsibilities

• Define and maintain target data engineering architecture for Data Science and AI workloads.
• Design patterns for moving data from AWS to Azure Data Science env; collaborate with IT & security teams.
• Own data engineering delivery: pipelines, transformations and curated datasets.
• Define reusable features, feature stores, lineage, versioning and monitoring.
• Establish production-grade MLOps/LLMOps: model deployment, versioning and retraining.
• Provide data engineering foundations for Generative/agentic AI: ingestion, indexing and governance.
• Ensure compliance with governance, privacy and security standards.
• Lead Data Engineers; translate roadmap into an engineering plan.
• Establish monitoring, support, incident management and cost optimisation; liaison with infra & governance teams.

Skills

Senior data engineering

Architecture

Python

SQL

MLOps

LLMOps

Data governance

Education

Degree in CS/IS/Data Eng/Math

Tools

Azure Data Factory

Azure Machine Learning

Azure Functions

Azure Storage

AWS (S3, Glue, Redshift)

Git CI/CD

Terraform

Databricks

Job description

DVT is seeking an experienced Data Engineering Lead to design, build and govern data and architecture for AI initiatives. You will own Azure- and AWS-based environments, develop governed data pipelines and collaborate with data scientists and engineers to productionise AI capabilities.

You will lead the data engineering team, ensure data quality and governance, and drive cost optimisation and incident management while aligning with security standards.