Data Engineer: Cloud + Spark Pipeline Architect

Network Recruitment; Cape Town; On-site; ZAR 700,000 - 1,100,000; Full time

Job summary

Network Recruitment seeks a skilled Data Engineer to design, build and optimise data pipelines in a growing financial services technology team. You will leverage Databricks, Apache Spark and PySpark to deliver robust data platforms, while collaborating with cross‑functional partners to align data solutions with business objectives.

The role emphasizes cloud-based data architectures, CI/CD practices, and strong SQL and Python capabilities to ensure scalable, reliable data products for consumers

Qualifications

• Degree or diploma in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• 3 to 5 years of experience in Data Engineering.
• Experience building and maintaining enterprise data solutions.
• Experience with Databricks in a production environment.
• Strong SQL development and optimisation experience.
• Exposure to cloud-based data platforms and services.

Responsibilities

• Build, maintain, and optimise data pipelines and workflows.
• Develop scalable data solutions using Databricks and Apache Spark.
• Design and implement end-to-end data flows and integration processes.
• Contribute to the development and maintenance of CI/CD pipelines.
• Collaborate with developers, analysts, and business stakeholders to deliver high-quality data products.
• Ensure data platforms are reliable, efficient, and aligned with business requirements.
• Support the implementation of modern data architecture principles.
• Continuously evaluate and adopt best practices in data engineering and cloud technologies.

Skills

Python PySpark

SQL

Azure Cloud

Databricks

Apache Spark

CI/CD

Backend Dev

Data modelling

ETL

Distributed computing

Education

Degree or diploma in Computer Science / IT / Engineering / Data Science

Tools

Azure

Microsoft Fabric

Medallion Architecture

Job description

Network Recruitment seeks a skilled Data Engineer to design, build and optimise data pipelines in a growing financial services technology team. You will leverage Databricks, Apache Spark and PySpark to deliver robust data platforms, while collaborating with cross‑functional partners to align data solutions with business objectives.

The role emphasizes cloud-based data architectures, CI/CD practices, and strong SQL and Python capabilities to ensure scalable, reliable data products for consumers