Senior Data Engineer

Rad Resources; Gauteng; On-site; ZAR 1,000,000 - 1,300,000; Full time

Job summary

Rad Resources seeks a seasoned Data Engineer to own the data platform and pipelines in Gauteng. You will design scalable ETL/ELT workflows, manage data warehouses, and partner with Cybersecurity and business teams to ensure quality, security, and compliance.

You will enable AI/ML capabilities with curated datasets and feature pipelines while driving automation and documentation across the data estate. This role emphasizes collaboration and reliable, scalable data solutions.

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Technology or related field.
• 5+ years' experience in data engineering, data platform development, or similar role.
• Hands-on experience with Databricks, Azure Data Factory, Azure SQL and PostgreSQL.
• Experience designing and supporting data warehouses, data marts, and ETL/ELT pipelines.
• Knowledge of Lakehouse architecture, Spark, and cloud-based data platforms.
• Understanding of data governance, data security, and compliance with POPIA/GDPR.
• Experience with Power BI, Microsoft Fabric/Synapse, or Azure data ecosystem is advantageous.
• Exposure to AI/ML data pipelines, Azure OpenAI, or MLOps.
• Familiarity with CI/CD, IaC (Bicep/Terraform), and Agile delivery methods.
• Relevant Microsoft Azure or Databricks certifications desirable.

Responsibilities

• Data Platform Management: Maintain availability, performance and cost efficiency of data platforms and pipelines.
• Data Engineering & Pipelines: Design and operate robust ETL/ELT pipelines and lakehouse workflows.
• Data Governance & Security: Ensure data quality, lineage, cataloguing, security and access control.
• AI & Analytics Enablement: Enable AI/ML workloads with curated datasets and feature pipelines.
• Security & Compliance: Ensure data estate meets cybersecurity and data-protection standards.
• Automation & Continuous Improvement: Develop automation to improve operational efficiency and platform reliability.
• Collaboration & Stakeholder Engagement: Build relationships with Development, Cybersecurity and business stakeholders.
• Reporting & Documentation Discipline: Produce monthly reports and maintain up-to-date technical documentation.

Skills

Data governance

Security & compliance

CI/CD

Agile delivery

Stakeholder engagement

Cloud experience

Education

Bachelor's degree in Computer Science, Data Engineering, Information Technology

Tools

Databricks

Azure Data Factory

Azure SQL

PostgreSQL

Power BI

Microsoft Fabric/Synapse

Job description

• Data Platform Management: Maintain the availability, performance, and cost efficiency of the Group’s data platform and pipelines through proactive monitoring and optimisation.
• Data Engineering & Pipelines: Design and operate robust ETL/ELT pipelines and lakehouse workflows that deliver trusted, well-modelled data to analytics and business consumers.
• Data Governance & Quality: Ensure data quality, lineage, cataloguing, security, and access control across all data assets, maintaining audit readiness.
• AI & Analytics Enablement: Enable AI and machine-learning workloads by providing curated datasets, feature pipelines, and the supporting platform capabilities.
• Security & Compliance: Ensure the data estate meets cybersecurity, patching, and data-protection standards with zero critical vulnerabilities.
• Automation & Continuous Improvement: Develop and implement automation and improvement initiatives that enhance operational efficiency and platform reliability.
• Collaboration & Stakeholder Engagement: Build strong relationships with the Development team, Cybersecurity, and business stakeholders to ensure alignment and effective delivery.
• Reporting & Documentation Discipline: Produce accurate monthly reports and maintain up-to-date technical documentation for all data platforms and pipelines.

Key Responsibilities

Data Engineering & Platform

• Design, build, and maintain scalable data pipelines and lakehouse solutions using Databricks, Azure Data Factory, Azure SQL, PostgreSQL, and Redis.
• Develop and optimize data warehouses, data marts, and ETL/ELT processes.
• Monitor platform performance, automate workflows, and maintain technical documentation.

AI, Analytics & Data Solutions

• Deliver trusted datasets to support reporting, analytics, and AI/ML initiatives.
• Collaborate with development teams to integrate data solutions into business applications.
• Ensure data platforms are scalable, performant, and cost-effective.

Data Governance & Security

• Own data governance, security, and compliance across the Group's data platform.
• Ensure compliance with POPIA, GDPR, and internal IT governance standards.
• Manage data access, lineage, cataloguing, encryption, and audit readiness.
• Partner with Cybersecurity on risk management, vulnerability remediation, and compliance reviews.

Stakeholder Collaboration

• Act as the primary data engineering contact within Group IT.
• Work closely with regional IT teams, R&D, and business stakeholders.
• Prepare operational reports, manage risks, and drive continuous improvement initiatives.

Qualifications & Experience

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
• 5+ years' experience in data engineering, data platform development, or a similar role.
• Strong hands-on experience with Databricks, Azure Data Factory, Azure SQL, and PostgreSQL.
• Experience designing and supporting data warehouses, data marts, and ETL/ELT pipelines.
• Knowledge of Lakehouse architecture, Spark, and cloud-based data platforms.
• Understanding of data governance, data security, and compliance with POPIA/GDPR.
• Experience with Power BI, Microsoft Fabric/Synapse, or the Azure data ecosystem is advantageous.
• Exposure to AI/ML data pipelines, Azure OpenAI, or MLOps is beneficial.
• Familiarity with CI/CD, Infrastructure-as-Code (Bicep/Terraform), and Agile delivery methodologies is an advantage.
• Relevant Microsoft Azure or Databricks certifications are highly desirable.