Cloud Engineer (Contract)

The Focus Group; Sandton; On-site; ZAR 800,000 - 1,200,000; Full time

Job summary

The Focus Group in Sandton, South Africa, is seeking an experienced Data Engineer to architect, build and maintain robust, scalable data pipelines, ingesting and transforming data from diverse sources using Ab Initio, Python/Scala, Spark and Databricks within Azure and on‑premises environments.

You will collaborate with security, governance and analytics teams to ensure data quality, lineage and secure real‑time access, implement automated testing and CI/CD for data workflows, and mentor junior

Job description

Responsibilities

• Architect, engineer and maintain robust scalable and fault tolerant data pipelines using technologies like Ab Initio, Python/Scala, Apache spark, Microsoft Fabric, Databricks for ingestion, transformation and delivery of structured, semi structured and unstructured data.
• Ensure efficient performance tuned extraction and loading of data into the enterprise Data Warehouse and Data Lake focusing on high availability, cost efficiency and reliability.
• Implement modern Data Ops practices including automation, testing and orchestration for both batch and low latency streaming data workflows.
• Collaborate with Software Engineers to design and implement consumption APIs and services that enable secure real time data access for business applications.
• Architect and manage the robust, hybrid cloud data platform utilising Azure Services (Microsoft Fabric, Ab initio, synapse, Databricks, SAS) and potentially on‑premises technologies (DB2 warehouse, Netezza, Denodo, Ab Initio).
• Establish and manage comprehensive observability for all data infrastructure components such as databases, data lakes, and data warehouses to meet strict service level objectives and aligned with architectural standards.
• Collaborate with Information Security, CISO, and Data Governance to enforce data security, policies and implement masking/anonymisation techniques to ensure compliance with data privacy regulations (POPIA).
• Lead the technical evaluation, proof of concept and the successful integration of cutting‑edge data engineering technologies and tools to enhance platform maturity and competitive advantage.
• Embed data quality as code by implementing automated, unit and end to end data validation, reconciliation, and auditing frameworks within the CI/CD data pipelines.
• Design and maintain the automated capture and maintenance of technical and operational metadata ensuring complete automated data lineage within the enterprise metadata hub (Ab Initio).
• Act as a technical guardian of the data integrity partnering with Data Governance to define and enforce master data management and data quality standards across all data products.
• Proactively manage data quality through regular data profiling, advanced issue analysis, and swift documented root cause resolution.
• Serve as trusted technical partner, collaborating with business stakeholders, Data Scientists, Analysts, and Architects to translate requirements into scalable data solutions.
• Drive the full data lifecycle of data products and end-to-end ownership of data engineering initiatives, ensuring timely delivery and alignment with business objectives.
• Provide Level 2/3 support for complex data issues, coordinating with cross‑functional teams to resolve incidents, ensuring a minimal mean time to resolution and clear communication.
• Contribute to sprint planning, backlog refinement, and agile delivery processes.
• Drive continuous optimisation of data engineering practices, standards, and processes to improve efficiency and performance.
• Mentor and guide junior engineers, providing coaching and technical support to build capability.
• Stay abreast of emerging technologies and industry trends to enhance data engineering maturity.
• Contribute to innovation initiatives that align with the bank's data-driven strategy.