Principal Data Engineer (Databricks)

Sabenza IT & Recruitment

Cape Town

On-site

ZAR 1,200,000 - 1,700,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Sabenza IT & Recruitment in Cape Town seeks a Principal Data Engineer to lead hands-on pipeline work and raise engineering standards across the team.

You will design robust Databricks pipelines, optimize performance and cost, and define trusted metrics while mentoring junior engineers and aligning data solutions with business needs.

Qualifications

  • At least three years of hands-on Databricks experience.
  • Proficiency with Spark Declarative Pipelines (SDP) and Metric Views.
  • Strong programming in Python, SQL and Spark.
  • Advanced data modelling and query optimisation.
  • Experience coaching junior developers.
  • Track record of managing and influencing business stakeholders.

Responsibilities

  • Design and build production-grade data pipelines on Databricks.
  • Optimize platform performance and cost by tuning compute, jobs and queries.
  • Define and maintain consistent business metrics and a semantic layer.
  • Integrate data from core systems (e.g., SAP S/4HANA) using Fivetran and HVR.
  • Set engineering standards across the team including CI/CD and governance.
  • Coach and mentor junior and mid-level engineers.
  • Partner with business stakeholders to shape practical solutions and deliver value.
  • Engage data owners to ensure data quality and fit for purpose.

Skills

Databricks
Spark
Python
SQL
Data modelling
Query optimisation
Coaching
Stakeholder management
Cost optimisation
Performance tuning
Team leadership

Education

Tertiary qualification in CS/IS/Engineering

Tools

Fivetran
HVR
SAP S/4HANA

Job description

Cape Town, South Africa | Posted on 10/11/2026

Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands‑on role, combining the building and optimization of production data pipelines with the technical leadership needed to raise engineering standards and develop the people around them.

The role functions as a key interface between business stakeholders and department heads, the wider data and analytics team, and the owners of source systems and data in Finance and IT.

Requirements
KEY RESPONSIBILITIES:
  • Design and build robust, production-grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP).
  • Work “under the hood” to optimise the platform for performance and cost tuning compute, jobs and queries,managing storage and table layout, and keeping platform spend under control.
  • Define and maintain consistent business metrics and a semantic layer (for example, using Metric Views) so that reporting is built on trusted, reusable definitions.
  • Integrate data from core business systems, including SAP S/4HANA, and other sources, using tools such as Fivetran (both SaaS connectors and HVR).
  • Set and uphold engineering standards across the team, including code quality, testing, documentation, CI/CD and data governance.
  • Coach and mentor junior and mid-level engineers, review their work and help them develop.
  • Partner with business stakeholders to understand their needs, shape practical solutions and ensure the platform delivers genuine value.
  • Engage data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiate data quality improvements where required.
REQUIRED SKILLS AND EXPERIENCE:
  • At least three years of hands‑on, day‑to‑day Databricks experience, covering both pipeline development and under‑the‑hood performance and cost optimisation.
  • Up to date with recent platform developments, such as Spark Declarative Pipelines (SDP) and Metric Views.
  • Strong proficiency in programming languages commonly used in data engineering, such as Python, SQL and Spark.
  • Advanced experience with data manipulation, data modelling, database design and query optimisation.
  • Experience managing or coaching junior developers.
  • A track record of managing and influencing business stakeholders.
KEY COMPETENCIES:
  • Combining deep, hands‑on engineering skill with sound judgement about cost, performance and long‑term maintainability.
  • Coaching, mentoring and raising the capability of less experienced engineers.
  • Collaborating, communicating confidently and influencing business stakeholders.
  • Breaking down complex technical concepts and explaining them simply to non‑technical audiences.
  • Staying current with a fast‑moving platform and bringing new capabilities into everyday practice.
  • Taking ownership and driving work independently, from concept through to production.
NICE TO HAVE:
  • Experience working with SAP S/4HANA data sets.
  • Experience with Fivetran, including both SaaS connectors and HVR.
QUALIFICATIONS:
  • A relevant tertiary qualification in Computer Science, Information Systems, Engineering or a related field, or equivalent practical experience.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Data Engineer (Databricks)
Principal Data Engineer (Databricks)

Ats Client • Cape Town

On-site
ZAR 900,000 - 1,500,000
Principal Data Engineer — Databricks & Platform Lead
Principal Data Engineer — Databricks & Platform Lead

Ats Client • Cape Town

On-site
ZAR 900,000 - 1,500,000
Data Engineer
Data Engineer

Parvana • Kraaifontein

Hybrid
ZAR 650,000 - 900,000
Databricks Consultant
Databricks Consultant

Sabenza IT & Recruitment • Cape Town

On-site
ZAR 600,000 - 900,000
Principal Data Engineer
Principal Data Engineer

Placements24 • Stellenbosch

Hybrid
ZAR 1,500,000 - 2,500,000
Health insurance
Retirement contributions
Life assurance
+4
Data Engineer
Data Engineer

Parvana • Bellville

Hybrid
ZAR 700,000 - 1,100,000
Senior Data Engineer
Senior Data Engineer

Sabenza IT & Recruitment • Pretoria

On-site
ZAR 800,000 - 1,200,000
Data Engineer
Data Engineer

Liquid Thought • Somerset West

On-site
ZAR 550,000 - 900,000
Senior Databricks Data Engineer
Senior Databricks Data Engineer

Hileya - Management Consulting • Sandton

On-site
ZAR 500,000 - 700,000
Data Engineer (MLOps / Analytics Focus)
Data Engineer (MLOps / Analytics Focus)

Blue Pearl PTY • Johannesburg

On-site
Confidential