Senior Kafka Engineer

Renaissance InfoSystems; Sydney; On-site; AUD 150,000 - 210,000; Full time

Job summary

Renaissance InfoSystems in Sydney seeks a Senior Kafka Administrator/Developer to design, build and operate enterprise event-streaming platforms on AWS and Kubernetes. You will implement reliable event-driven solutions, guide teams on design choices and drive platform governance.

The role requires hands-on Kafka administration, integration development in Java/Spring Boot or Python, and strong cloud-native operations across production environments.

Qualifications

• Hands-on Kafka administration in enterprise environments.
• Experience with AWS services and EKS orchestration.
• Proficient in Java/Spring Boot or Python integration.
• Strong DevOps mindset with CI/CD and GitOps.
• Knowledge of security, TLS, certificates and least-privilege access.
• Ability to design reliable, scalable event-driven platforms.

Responsibilities

• Administer and optimize Kafka platforms, brokers, topics and connectors.
• Develop producers/consumers and integration components in Java/Python.
• Deploy and operate on Kubernetes and Amazon EKS with Helm and CI/CD.
• Ensure platform reliability, performance, patching and recovery.
• Implement monitoring, alerts and dashboards for health and lag.
• Lead governance, security controls and incident response.
• Mentor engineers on topic design and operational standards.
• Document architecture, runbooks and platform improvements.

Skills

Kafka administration

Kubernetes

AWS / EKS

Java / Spring Boot

Python

CI/CD / GitOps

Observability

Linux / Networking

Security / TLS

Education

Degree in IT / CS

Certifications (AWS/Kubernetes)

Tools

Jenkins

Argo CD

Terraform

CloudFormation

Docker

Job description

The Senior Kafka Administrator / Developer will design, build, operate and improve enterprise Kafka and event-streaming platforms across AWS and Kubernetes, including Amazon EKS. The role requires strong hands-on Kafka administration, integration development, cloud-native operations and production support experience.

The role works closely with application, integration, cloud, security and operations teams to deliver reliable event-driven solutions, resolve complex issues, automate repeatable tasks and support platform lifecycle, governance and service improvement activities.

Key Responsibilities

• Administer and support Kafka platforms, including brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and related services.
• Design and develop Kafka producers, consumers, stream-processing components and reusable integration patterns using Java, Spring Boot, Python or similar technologies.
• Deploy, operate and improve Kafka and supporting components on Kubernetes and Amazon EKS using Helm, CI/CD, GitOps and approved cloud-native patterns.
• Manage platform reliability, resilience, capacity, performance, patching, upgrades, backup, recovery and lifecycle activities across production and non-production environments.
• Troubleshoot complex issues across applications, brokers, connectors, schemas, networking, storage, certificates and cloud infrastructure.
• Implement monitoring, alerting and dashboards for broker health, throughput, latency, replication, disk usage, consumer lag and connector health.
• Apply security and governance controls, including TLS, authentication, authorisation, ACLs, secrets, certificates, least-privilege access and vulnerability remediation.
• Support incidents, problems, changes and releases, including root-cause analysis, runbook improvement and automation of recurring operational tasks.
• Guide application teams on topic design, partitioning, message contracts, schema evolution, retries, dead-letter handling, idempotency and supportability.
• Maintain architecture, configuration, operational, recovery and support documentation.

Technical Leadership

• Act as the senior technical point of contact for Kafka platform design, engineering and operational matters.
• Mentor engineers, review technical work and promote consistent engineering and operational standards.
• Lead planning for upgrades, migrations, resilience improvements and platform modernisation.
• Communicate risks, dependencies, options and recommendations clearly to technical and business stakeholders.

Key Skills & Experience

Technical Experience

• Strong hands-on experience administering Apache Kafka or Confluent Platform in complex enterprise environments.
• Solid understanding of Kafka brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and MirrorMaker.
• Experience developing Kafka producers, consumers and integration services using Java, Spring Boot, Python or equivalent technologies.
• Hands-on experience with Kubernetes and Amazon EKS, including Docker, Helm, storage, networking, ingress, secrets and identity integration.
• Working knowledge of AWS services such as EKS, EC2, S3, IAM, CloudWatch, Route 53 and core networking services.
• Experience with CI/CD, GitOps and Infrastructure as Code tools such as Jenkins, Argo CD, Terraform or CloudFormation.
• Strong Linux, networking, TLS, DNS, certificate and distributed-systems troubleshooting capability.
• Experience with observability tools such as Prometheus, Grafana, CloudWatch, Dynatrace, Splunk or equivalent.

Preferred Qualifications

• Degree in Information Technology, Engineering, Computer Science, or related discipline.
• AWS, Azure, ITIL, Kubernetes, Database, or relevant technology certifications.
• Experience managing large-scale enterprise or government technology environments.
• Experience with hybrid cloud, platform modernization, and operational transformation programs.