Databricks & Cloud Data Platform Lead — Cape Town

DVT; Cape Town; On-site; ZAR 900,000 - 1,500,000; Full time

Job summary

DVT, based in Capetown, is hiring a Senior Data Engineer with Databricks expertise to design and deliver modern, scalable data platforms for enterprise clients. You will work closely with architects, analysts, data scientists, and client stakeholders to create secure, high-performing analytics platforms across cloud environments.

You will lead design efforts for lakehouse architectures, implement governance, and mentor junior engineers while delivering end-to-end data solutions using PySpark,

Qualifications

• 8+ years in Data Engineering projects.
• 3+ years of hands-on Databricks delivery experience.
• Deep expertise in Python and PySpark.
• Strong understanding of Medallion and Lakehouse architectures.
• Experience building live Databricks dashboards using SQL.
• Proficient with Azure data services (Synapse, ADF, OneLake, Purview).
• Experience with AWS data platforms (Glue, S3, Redshift).
• Consulting or client-facing environment experience.
• Bachelor’s degree in computer science, engineering, information systems, or equivalent.

Responsibilities

• Design and implement scalable Databricks-based data platforms.
• Build ETL/ELT pipelines for batch and streaming workloads.
• Lead data migration to cloud-based lakehouse architectures.
• Develop data processing solutions using PySpark, Python, and SQL.
• Implement Delta Lake, Unity Catalog, and Purview governance.
• Design lakehouse and medallion architectures for enterprises.
• Optimize Spark workloads for performance, cost, and scalability.
• Create Databricks Workflows, Airflow, or similar orchestration.
• Collaborate with stakeholders to define requirements.
• Mentor junior engineers and contribute to best practices.
• Produce architecture artefacts and reusable assets.

Skills

Databricks

Cloud data platforms

Mentoring

Education

Bachelor's degree in CS/Engineering

Tools

Databricks

Azure

AWS

Delta Lake

Unity Catalog

Purview

Airflow

ADF

Job description

DVT, based in Capetown, is hiring a Senior Data Engineer with Databricks expertise to design and deliver modern, scalable data platforms for enterprise clients. You will work closely with architects, analysts, data scientists, and client stakeholders to create secure, high-performing analytics platforms across cloud environments.

You will lead design efforts for lakehouse architectures, implement governance, and mentor junior engineers while delivering end-to-end data solutions using PySpark,