Kafka Specialist

Datonomy Solutions (Pty) Ltd; Sandton; Hybrid; ZAR 900,000 - 1,200,000; Full time

Job summary

Datonomy Solutions (Pty) Ltd is seeking an experienced Technology Specialist – Kafka to design, administer and optimise enterprise Kafka platforms for mission-critical banking applications.

The role emphasizes stability, scalability, resilience and security of Kafka environments, with automation, IaC and cloud adoption at the core of the responsibilities.

Qualifications

• 6+ years in Systems/Platform Engineering or Infrastructure Engineering.
• 4+ years hands-on Kafka administration in enterprise environments.
• Experience supporting highly available distributed systems.
• Experience automating infrastructure and operational processes.
• DevOps experience in enterprise settings.

Responsibilities

• Kafka Platform Administration: Install, configure, upgrade, patch and maintain enterprise Kafka clusters.
• Platform Engineering: Design highly available Kafka platforms across environments and implement resilient architectures.
• DevOps & Automation: Develop automation for infrastructure as code, platform provisioning, automated deployments, configuration management, CI/CD, and environment standardisation.
• Linux Platform Administration: Provide specialist Linux administration, performance tuning, storage and network troubleshooting, and capacity management.
• Kafka Security: Design and maintain TLS/SSL, SASL, ACLs, certificate lifecycles, and secure client connectivity.
• Cloud Engineering: Support deployments on Azure with networking, VMs, managed services, storage, identity integration, and hybrid deployments.
• Monitoring & Observability: Build health dashboards, alerting, log aggregation, and distributed tracing across Kafka platforms.
• Incident & Problem Management: Support production platforms within SLA, lead investigations, and drive stability improvements.
• Collaboration: Work with Platform Engineers, DevOps, Infrastructure, Cloud, Security, and Architecture teams.

Skills

Apache Kafka

Confluent Platform

Confluent Cloud

Linux Administration

Terraform

Ansible

Python

CI/CD

Azure DevOps

Git

Tools

Prometheus

Grafana

ELK Stack

Splunk

Azure Monitor

Azure

Terraform

Job description

We are seeking an experiencedTechnology Specialist – Kafkato design, administer, automate, secure, and optimise enterprise Kafka platforms supporting mission-critical banking applications.

This role is responsible for ensuring the stability, scalability, resilience, and security of Kafka environments while driving automation, DevOps best practices, and cloud adoption.

The successful candidate will combine deep Kafka platform engineering expertise with Linux administration, Infrastructure-as-Code, observability, and cloud engineering to deliver highly available event streaming infrastructure that supports modern banking systems and real-time data integration.

Key Responsibilities

Kafka Platform Administration

• Install, configure, upgrade, patch and maintain enterprise Kafka clusters.
• Administer Confluent Platform and/or Confluent Cloud environments.
• Manage brokers, topics, partitions, replication and consumer groups.
• Optimise Kafka cluster performance, throughput and latency.
• Perform capacity planning and scaling of Kafka infrastructure.
• Troubleshoot broker failures, replication issues and performance bottlenecks.
• Support production incidents and perform root cause analysis.

Platform Engineering

• Design highly available Kafka platforms across multiple environments.
• Implement resilient architectures for business-critical messaging workloads.
• Configure multi-broker clusters with appropriate replication and fault tolerance.
• Support disaster recovery and high availability strategies.
• Maintain platform documentation and operational runbooks.

DevOps & Automation

Develop and maintain automation to reduce manual operational effort.

Responsibilities include:

• Infrastructure as Code
• Platform provisioning
• Automated deployments
• Configuration management
• CI/CD integration
• Environment standardisation

Typical technologies include:

• Terraform
• Ansible
• Python
• Bash
• Azure DevOps
• Git

Linux Platform Administration

Provide specialist Linux administration supporting Kafka infrastructure.

Responsibilities include:

• Performance tuning
• Storage optimisation
• Network troubleshooting
• Service management
• Operating system maintenance
• Capacity management
• Log analysis

Kafka Security

Design and maintain secure Kafka environments.

Responsibilities include:

• TLS/SSL encryption
• SASL authentication
• ACL administration
• Certificate lifecycle management
• Identity integration
• Secure client connectivity
• Compliance with enterprise security standards

Cloud Engineering

Support Kafka deployments across cloud platforms, with a focus on Microsoft Azure.

Responsibilities include:

• Azure networking
• Virtual Machines
• Managed services
• Storage
• Identity integration
• Hybrid deployments
• Cloud security
• Platform optimisation

Experience with Confluent Cloud is highly advantageous.

Monitoring & Observability

Build comprehensive monitoring and operational visibility across Kafka platforms.

Responsibilities include:

• Health monitoring
• Performance dashboards
• Alerting
• Capacity monitoring
• Log aggregation
• Distributed tracing
• Incident diagnostics

Typical tools include:

• Prometheus
• Grafana
• ELK Stack
• Splunk
• Azure Monitor

Incident & Problem Management

• Support production platforms within SLA.
• Lead technical investigations.
• Perform root cause analysis.
• Recommend permanent fixes.
• Participate in major incident resolution.
• Drive platform stability improvements.

Collaboration

Work closely with:

• Platform Engineers
• DevOps Engineers
• Infrastructure Teams
• Cloud Engineers
• Security Teams
• Application Developers
• Architecture Teams
• Operations Teams

Minimum Requirements

Experience

• 6+ years' experience in Systems Engineering, Platform Engineering or Infrastructure Engineering.
• 4+ years' hands-on Kafka administration.
• Enterprise production support experience.
• Experience supporting highly available distributed systems.
• Experience automating infrastructure and operational processes.
• Experience working within DevOps environments.

Required Technical Skills

Kafka

• Apache Kafka
• Confluent Platform
• Confluent Cloud
• Broker administration
• Topic management
• Consumer groups
• Partition management
• Replication
• Cluster upgrades
• Performance tuning
• Capacity planning

Automation

• Python
• Bash
• Terraform
• Ansible
• CI/CD Pipelines
• Azure DevOps
• Git

Operating Systems

• Linux Administration
• Shell scripting
• Performance tuning
• System troubleshooting

Cloud

• Microsoft Azure
• Azure networking
• Azure infrastructure
• Azure Monitor
• Hybrid cloud environments

Kafka Security

• TLS/SSL
• SASL
• ACLs
• Certificates
• Encryption
• Authentication
• Authorisation

Monitoring & Observability

• Prometheus
• Grafana
• ELK Stack
• Splunk
• Azure Monitor
• Log analysis
• Alerting
• Performance monitoring

Preferred Experience

• Banking or Financial Services experience.
• Confluent Cloud implementation experience.
• Enterprise messaging platforms.
• Event-driven architecture.
• Kubernetes or container platforms.
• Docker.
• Site Reliability Engineering (SRE).
• Platform Engineering.
• Infrastructure as Code at enterprise scale.

Nice-to-Have Certifications

• Confluent Certified Developer or Administrator
• Microsoft Azure Administrator
• Microsoft Azure Solutions Architect
• Azure DevOps Engineer
• Red Hat Certified System Administrator (RHCSA)
• Terraform Associate
• Kubernetes Certifications (CKA/CKAD)

Ideal Candidate Profile

The ideal candidate is a senior Platform or Systems Engineer with deep expertise inApache KafkaandConfluent Platform, capable of administering large-scale event streaming platforms in enterprise environments.

They should have strong Linux administration skills, experience automating infrastructure throughTerraform, Ansible, Python, and CI/CD pipelines, and a solid understanding ofKafka security, cloud infrastructure (particularlyAzure), and modern observability practices. Experience within a regulated environment such as banking or financial services would be highly advantageous.