Kafka Developer

Fuku; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A tech company in Singapore is looking for a Kafka Developer to design and maintain Kafka-based integration solutions. The ideal candidate will have strong skills in Java and Kubernetes, with a focus on ensuring high availability and reliability for distributed systems. Responsibilities include managing Kafka clusters, developing integration solutions, and collaborating with teams across the organization. This role requires expertise in monitoring tools and automation practices.

Qualifications

• In-depth knowledge of Apache Kafka architecture including topics and partitions.
• Strong proficiency in programming with Java or Python for application development.
• Solid understanding of Kubernetes for managing containers and resources.
• Hands-on experience with API Gateway technologies.

Responsibilities

• Design, implement, and manage Kafka-based messaging and streaming solutions.
• Administer and maintain Kafka clusters ensuring reliability and fault tolerance.
• Monitor Kafka clusters proactively identifying and resolving issues.
• Collaborate with cross-functional teams supporting integration and DevOps initiatives.

Skills

Kafka expertise

Java proficiency

Kubernetes expertise

API Gateway knowledge

DevOps practices understanding

Scripting skills

Monitoring tools experience

Distributed systems knowledge

Tools

Apache Kafka

Kubernetes

Grafana

Kafka Manager

Job description

Job Title: Kafka Developer

We are seeking a highly skilled Java – Kafka Integration Engineer with strong expertise in Apache Kafka, Kubernetes, and distributed systems. The ideal candidate will be responsible for designing, developing, deploying, and maintaining scalable Kafka-based integration solutions while ensuring high availability, performance, and security across distributed environments.

Key Responsibilities

• Design, implement, and manage Kafka-based messaging and streaming solutions.
• Administer and maintain Kafka clusters, ensuring reliability, scalability, and fault tolerance.
• Develop Kafka producers, consumers, and stream processing applications using Java (or Python where applicable).
• Deploy and manage Kafka and related components in Kubernetes environments.
• Design and implement Custom Resource Definitions (CRDs) and develop controllers to manage Kubernetes resources.
• Integrate Kafka with API Gateway solutions to securely expose and manage APIs.
• Automate infrastructure provisioning, deployment, and operational tasks using scripts and CI/CD pipelines.
• Monitor Kafka clusters and applications, proactively identifying and resolving performance or reliability issues.
• Collaborate with cross-functional teams to support integration, DevOps, and platform engineering initiatives.

Required Skills & Qualifications

1. Kafka Expertise (Administration & Development)

• In-depth knowledge of Apache Kafka architecture, including topics, partitions, brokers, producers, and consumers.
• Strong experience with Kafka APIs:
  • Producer API
  • Consumer API
  • Kafka Streams API
  • Kafka Connect API

2. Programming Languages

• Strong proficiency in Java (preferred) or Python for Kafka application development.
• Familiarity with scripting languages such as Bash or PowerShell for automation and operational tasks.

3. Kubernetes Expertise

• In-depth understanding of Kubernetes, including:
  • Operators and providers
  • Container management
  • Component upgrades and lifecycle management
  • Debugging Kubernetes resources and managed objects
• Proficiency with Custom Resource Definitions (CRDs).
• Ability to design and implement new CRDs and develop controllers to manage them effectively.

4. API Gateway Knowledge

• Hands-on experience with at least one API Gateway technology such as:
• Kong
• Apigee
• Gravitee

5. DevOps & CI/CD

• Solid understanding of DevOps practices.
• Experience with CI/CD pipelines to automate build, test, and deployment processes.

6. Distributed Systems

• Strong understanding of distributed systems concepts, including:
• Replication
• Partitioning
• Fault tolerance
• High availability

7. Monitoring & Logging

• Experience with monitoring and observability tools such as Grafana, Kafka Manager, or similar.
• Familiarity with logging frameworks to ensure system health, performance, and reliability.

8. Automation & Scripting

• Strong automation and scripting skills for managing Kafka infrastructure, deployments, and operational workflows.