Data Platform Engineer - Pipelines, Lakehouse & Quality

BETSoftware

Johannesburg

On-site

ZAR 540,000 - 900,000

Full time

7 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

BETSoftware is seeking a data engineer to design and maintain batch and near-real-time data pipelines that support analytics and operations. You will work across ingestion, transformation, storage, and serving layers of the data platform, using SQL, Python, and modern tooling.

You will partner with BI, analytics, software engineering, and product teams to translate requirements into scalable engineering solutions, modernise legacy workflows, and ensure data quality and reliability across

Qualifications

  • Degree or diploma in IT, Computer Science, Engineering, or a related technical discipline.
  • 2 to 4 years of experience in data engineering, ETL/ELT development, or data platform engineering.
  • Hands-on SQL experience, including query optimisation, indexing, and performance tuning.
  • Experience building and maintaining batch data pipelines in production environments.
  • Exposure to data platform concepts such as data warehouse, data lake architecture, or object storage.
  • Experience with orchestration platforms such as Airflow, Dagster, or similar.

Responsibilities

  • Data Platform Engineering: Build, maintain, and improve batch and near-real-time data pipelines that support analytics and operational use cases.
  • Contribute to ingestion, transformation, storage, and serving layers within the enterprise data platform.
  • Develop reliable data processing logic using SQL, Python, and related tooling.
  • Help modernise legacy data workflows into more scalable and supportable patterns.
  • Partner with BI, analytics, software engineering, product, and business stakeholders to understand and support data use cases.
  • Translate business and platform requirements into scalable, well-engineered technical solutions.

Skills

Data engineering
SQL
Python
ETL/ELT
Batch data pipelines
Data warehousing
Airflow
Dagster
Kafka
Spark
Flink
CI/CD tooling
Kubernetes
OpenShift
Git
Object storage

Education

Degree or diploma in IT, Computer Science, Engineering, or a related technical discipline

Tools

Airflow
Dagster
Kafka/Redpanda
Spark
Flink
Kubernetes
OpenShift
Git
CI/CD tooling
Object storage
Data lake / warehouse

Job description

BETSoftware is seeking a data engineer to design and maintain batch and near-real-time data pipelines that support analytics and operations. You will work across ingestion, transformation, storage, and serving layers of the data platform, using SQL, Python, and modern tooling.

You will partner with BI, analytics, software engineering, and product teams to translate requirements into scalable engineering solutions, modernise legacy workflows, and ensure data quality and reliability across

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Cloud Data Platform Engineer: Lakehouse & Pipelines
Cloud Data Platform Engineer: Lakehouse & Pipelines

Decision Inc. • Johannesburg

On-site
ZAR 600,000 - 900,000
Cloud Data Engineer - Pipelines, Lakehouse & Analytics
Cloud Data Engineer - Pipelines, Lakehouse & Analytics

TFG (The Foschini Group) • Wes-Kaap

On-site
ZAR 900,000 - 1,200,000
Senior Data Platform Engineer – Lakehouse & Cloud
Senior Data Platform Engineer – Lakehouse & Cloud

Blue Pearl PTY • Johannesburg

On-site
ZAR 800,000 - 1,200,000
None
Databricks Lakehouse Engineer: Build & Tune Pipelines
Databricks Lakehouse Engineer: Build & Tune Pipelines

Sabenza IT & Recruitment • Eersterivier

On-site
ZAR 900,000 - 1,300,000
Lakehouse Data Engineer | Azure Databricks & AI-ready Data
Lakehouse Data Engineer | Azure Databricks & AI-ready Data

PBT Group • Johannesburg

On-site
ZAR 600,000 - 1,200,000
Senior Data Engineer / Platform Lead
Senior Data Engineer / Platform Lead

Ats Client • Pretoria

On-site
ZAR 900,000 - 1,300,000
Senior AWS Data Engineer - Databricks & Lakehouse Architect
Senior AWS Data Engineer - Databricks & Lakehouse Architect

PBT Group • Johannesburg

On-site
ZAR 900,000 - 1,300,000
Data Engineer
Data Engineer

Network Contracting • Durban

On-site
ZAR 600,000 - 1,000,000
Data Engineer
Data Engineer

Indsafri • South Africa

On-site
ZAR 600,000 - 900,000
Intermediate Data Engineer
Intermediate Data Engineer

BETSoftware • Johannesburg

On-site
ZAR 540,000 - 900,000