Azure Cloud Data Engineer Real-Time Pipelines & Data Lakes

DeARX; Sandton; On-site; ZAR 650,000 - 1,000,000; Full time

Job summary

DeARX is seeking an experienced Cloud Data Engineer in Sandton to design, build and optimise scalable cloud data platforms. You will create robust data pipelines for batch and real-time processing using Azure services and Databricks, while ensuring data quality, governance and security.

Ideal candidates have 2–5 years in data engineering or BI, hands-on experience with Azure Data Factory, PySpark and SQL, and a solid background in cloud data architectures.

Qualifications

• Bachelor's degree or equivalent in IT/CS.
• 2–5 years' experience in data engineering, BI, or software engineering.
• Experience building enterprise cloud data solutions.
• Familiarity with Agile delivery environments.

Responsibilities

• Design scalable cloud data pipelines for batch and real-time processing.
• Build robust ETL/ELT solutions using Azure and modern data engineering tech.
• Develop and maintain Data Lakes and Data Warehouses.
• Work with Microsoft Fabric, Azure Data Factory, Databricks and Azure services.
• Develop data ingestion frameworks for structured, semi-structured, and unstructured data.
• Implement DataOps, CI/CD pipelines and workflow orchestration.
• Ensure data quality, governance, lineage and metadata management.
• Collaborate with engineers, data scientists, architects and stakeholders to deliver data products.
• Support cloud security, data privacy and POPIA compliance.
• Monitor and optimise cloud data platforms for performance and reliability.
• Contribute to platform innovation and automation.

Skills

Azure

Azure Data Factory

Databricks

Python

PySpark

Apache Spark

SQL

Data Warehousing

Data Lakes

DataOps

CI/CD

IaC

Kafka or Azure Event Hub

Real-time processing

Metadata Management

Data Quality & Governance

Education

Bachelor's Degree in Information Technology, Computer Science or related field

Tools

Azure Synapse

Ab Initio

SAS

Denodo

DB2 Warehouse

Netezza

Job description

DeARX is seeking an experienced Cloud Data Engineer in Sandton to design, build and optimise scalable cloud data platforms. You will create robust data pipelines for batch and real-time processing using Azure services and Databricks, while ensuring data quality, governance and security.

Ideal candidates have 2–5 years in data engineering or BI, hands-on experience with Azure Data Factory, PySpark and SQL, and a solid background in cloud data architectures.