Kafka Administrator

Kezan Consulting; Gurugram District, Bengaluru; Hybrid; INR 900,000 - 1,500,000; Full time

Job summary

Kezan Consulting in Gurugram, India is seeking a Kafka Administrator to manage production Kafka clusters. You will handle brokers, topics, partitions, replication, and consumer groups while monitoring health and performance.

You will troubleshoot issues across producers, consumers, and brokers, and implement security and reliability measures. The role requires strong Linux/Unix and SQL fundamentals, plus scripting skills (Shell/Python) to automate tasks and maintain documentation.

Qualifications

• 4+ years of hands-on experience in Kafka Administration.
• Strong knowledge of Apache Kafka architecture.
• Experience with Kafka Brokers, Topics, Partitions, Replication and Consumer Groups.
• Strong understanding of Producer/Consumer architecture.
• Hands-on experience with Kafka monitoring and troubleshooting.

Responsibilities

• Administer and maintain Apache Kafka clusters in production environments.
• Manage Kafka brokers, topics, partitions, replicas, consumer groups, and configurations.
• Monitor Kafka cluster health, performance, throughput, lag, and resource utilization.
• Troubleshoot issues related to brokers, producers, consumers, replication, and message delivery.
• Perform Kafka cluster upgrades, configuration changes, patching, and maintenance.
• Monitor and resolve Consumer Lag, Under Replication, ISR, and broker availability issues.
• Configure and manage Kafka Connect, producers, consumers, and connectors.
• Implement and manage Kafka security including SSL/TLS, SASL, authentication, and authorization/ACLs.
• Perform capacity planning and Kafka performance tuning.
• Manage partitioning and replication strategies for high availability and fault tolerance.
• Handle production incidents and perform root cause analysis (RCA).
• Create and maintain operational documentation, SOPs, and troubleshooting guides.
• Collaborate with DevOps, Application, Cloud, and Development teams for Kafka-related issues.
• Automate routine administration and monitoring activities using scripting/tools.

Skills

Kafka Administration

Kafka Architecture

Kafka Connect

Linux/Unix

SQL Fundamentals

Shell/Python

Troubleshooting

Communication

Job description

Key Responsibilities

• Administer and maintain Apache Kafka clusters in production environments.
• Manage Kafka brokers, topics, partitions, replicas, consumer groups, and configurations.
• Monitor Kafka cluster health, performance, throughput, lag, and resource utilization.
• Troubleshoot issues related to brokers, producers, consumers, replication, and message delivery.
• Perform Kafka cluster upgrades, configuration changes, patching, and maintenance.
• Monitor and resolve Consumer Lag, Under Replication, ISR, and broker availability issues.
• Configure and manage Kafka Connect, producers, consumers, and connectors.
• Implement and manage Kafka security including SSL/TLS, SASL, authentication, and authorization/ACLs.
• Perform capacity planning and Kafka performance tuning.
• Manage partitioning and replication strategies for high availability and fault tolerance.
• Handle production incidents and perform root cause analysis (RCA).
• Create and maintain operational documentation, SOPs, and troubleshooting guides.
• Collaborate with DevOps, Application, Cloud, and Development teams for Kafka-related issues.
• Automate routine administration and monitoring activities using scripting/tools.

Must-Have Skills

• 4+ years of hands-on experience in Kafka Administration.
• Strong knowledge of Apache Kafka architecture.
• Experience with Kafka Brokers, Topics, Partitions, Replication and Consumer Groups.
• Strong understanding of Producer/Consumer architecture.
• Hands-on experience with Kafka monitoring and troubleshooting.
• Knowledge of Consumer Lag, ISR, Under Replication, Leader/Follower concepts.
• Experience with Kafka security SSL/TLS, SASL, ACLs.
• Experience with Kafka Connect and connectors.
• Good knowledge of Linux/Unix administration.
• Strong SQL and database fundamentals.
• Experience with scripting using Shell/Python is preferred.
• Good troubleshooting, analytical, and communication skills.