Kafka Developer

XPT Software Australia; Sydney; On-site; AUD 180,000 - 240,000; Full time

Job summary

XPT Software Australia, Sydney, is seeking a Senior Kafka Administrator/Developer to design, build, operate and improve enterprise Kafka platforms across AWS and Kubernetes, including Amazon EKS. You will work on brokers, topics, replication and integration services using Java, Spring Boot or Python.

The role requires strong hands-on Kafka administration, cloud-native operations and production support experience, with focus on reliability, security and scalable streaming architectures across

Qualifications

• Hands-on experience administering Apache Kafka or Confluent Platform in enterprise environments.
• Knowledge of Kafka brokers, topics, partitions, replication and consumer groups.
• Experience developing Kafka producers/consumers using Java, Spring Boot or Python.
• Kubernetes with AWS EKS, Docker, Helm, networking, secrets, identity integration.
• Experience with AWS services (EKS, EC2, S3, IAM, CloudWatch).
• Experience with CI/CD and IaC tools like Jenkins, Argo CD, Terraform or CloudFormation.

Responsibilities

• Administer and support Kafka platforms, including brokers, topics, partitions and replication.
• Design and develop Kafka producers/consumers and integration components.
• Deploy and manage Kafka on Kubernetes and EKS using Helm and CI/CD pipelines.
• Ensure platform reliability, performance, patching, backups and recovery.
• Troubleshoot complex issues across applications, brokers, schemas and cloud infra.
• Implement monitoring and dashboards for broker health and throughput.
• Apply security controls including TLS, authentication, ACLs and secrets.

Skills

Kafka administration

Kafka Connect

Kubernetes

AWS EKS

Java

Python

CI/CD

Security & TLS

Monitoring & observability

Troubleshooting

Education

Bachelor's degree in IT/Engineering/CS

Tools

Jenkins

Argo CD

Terraform

CloudFormation

Job description

Role Summary

The Senior Kafka Administrator / Developer will design, build, operate and improve enterprise Kafka and event-streaming platforms across AWS and Kubernetes, including Amazon EKS. The role requires strong hands-on Kafka administration, integration development, cloud-native operations and production support experience.

Role Summary

The Senior Kafka Administrator / Developer will design, build, operate and improve enterprise Kafka and event-streaming platforms across AWS and Kubernetes, including Amazon EKS. The role requires strong hands-on Kafka administration, integration development, cloud-native operations and production support experience.

Key Responsibilities

• Administer and support Kafka platforms, including brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and related services.
• Design and develop Kafka producers, consumers, stream-processing components and reusable integration patterns using Java, Spring Boot, Python or similar technologies.
• Deploy, operate and improve Kafka and supporting components on Kubernetes and Amazon EKS using Helm, CI/CD, GitOps and approved cloud-native patterns.
• Manage platform reliability, resilience, capacity, performance, patching, upgrades, backup, recovery and lifecycle activities across production and non-production environments.
• Troubleshoot complex issues across applications, brokers, connectors, schemas, networking, storage, certificates and cloud infrastructure.
• Implement monitoring, alerting and dashboards for broker health, throughput, latency, replication, disk usage, consumer lag and connector health.
• Apply security and governance controls, including TLS, authentication, authorisation, ACLs, secrets, certificates, least-privilege access and vulnerability remediation.
• Support incidents, problems, changes and releases, including root-cause analysis, runbook improvement and automation of recurring operational tasks.
• Guide application teams on topic design, partitioning, message contracts, schema evolution, retries, dead-letter handling, idempotency and supportability.
• Maintain architecture, configuration, operational, recovery and support documentation.

Technical Leadership

• Act as the senior technical point of contact for Kafka platform design, engineering and operational matters.
• Mentor engineers, review technical work and promote consistent engineering and operational standards.
• Lead planning for upgrades, migrations, resilience improvements and platform modernisation.
• Communicate risks, dependencies, options and recommendations clearly to technical and business stakeholders.

Key Skills & Experience

• Strong hands-on experience administering Apache Kafka or Confluent Platform in complex enterprise environments.
• Solid understanding of Kafka brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and MirrorMaker.
• Experience developing Kafka producers, consumers and integration services using Java, Spring Boot, Python or equivalent technologies.
• Hands-on experience with Kubernetes and Amazon EKS, including Docker, Helm, storage, networking, ingress, secrets and identity integration.
• Working knowledge of AWS services such as EKS, EC2, S3, IAM, CloudWatch, Route 53 and core networking services.
• Experience with CI/CD, GitOps and Infrastructure as Code tools such as Jenkins, Argo CD, Terraform or CloudFormation.
• Strong Linux, networking, TLS, DNS, certificate and distributed-systems troubleshooting capability.
• Experience with observability tools such as Prometheus, Grafana, CloudWatch, Dynatrace, Splunk or equivalent.

Preferred Qualifications

• Degree in Information Technology, Engineering, Computer Science, or related discipline.
• AWS, Azure, ITIL, Kubernetes, Database, or relevant technology certifications.
• Experience managing large-scale enterprise or government technology environments.
• Experience with hybrid cloud, platform modernization, and operational transformation programs.