Software Engineer

Wissen Technology; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Wissen Technology is seeking an experienced Kafka Administrator with strong Python scripting skills to join our engineering team in Bengaluru. You will manage and secure Kafka environments across development, testing, and production, focusing on high availability, reliability, and scalability.

Responsibilities include tuning performance, monitoring health and lag, and implementing automated tooling. The role requires 3–5 years of hands-on Kafka administration and strong Linux/Unix proficiency.

Qualifications

• 3–5 years hands-on Apache Kafka administration and production support.
• Deep knowledge of Kafka architecture: brokers, topics, partitions, replicas, consumers.
• Experience with monitoring, alerting, and performance troubleshooting of Kafka.
• Proficient in Python scripting for automation and tooling.
• Strong Linux/Unix command-line skills and shell scripting.
• Familiarity with security: ACLs, authentication, SSL/TLS.
• Experience with Kafka Connect/Kafka Streams and Confluent Platform is a plus.

Responsibilities

• Administer, monitor, and maintain Kafka clusters across development, testing, and production environments.
• Manage Kafka brokers, topics, partitions, replication factors, consumer groups, and configurations.
• Monitor Kafka cluster health, broker performance, replication status, consumer lag, throughput, and disk utilization.
• Troubleshoot Kafka issues related to broker failures, under-replicated partitions, consumer lag, rebalancing, connectivity, performance, and message delivery.
• Perform Kafka performance tuning and capacity planning based on workload and growth requirements.
• Configure and manage Kafka retention, partitioning, replication, and other topic-level settings.
• Manage Kafka security including authentication, authorization, ACLs, and secure communication.
• Support Kafka cluster installation, configuration, upgrades, patching, and migration activities.
• Implement and maintain monitoring and alerting for Kafka infrastructure.
• Develop Python scripts and automation tools for Kafka administration, monitoring, health checks, reporting, and operational activities.
• Automate routine administrative and support activities using Python and shell scripting.
• Analyze Kafka logs and metrics to identify issues and perform root cause analysis.
• Support Kafka producers and consumers by troubleshooting connectivity, configuration, throughput, and consumer-group related issues.
• Collaborate with Application Development, DevOps, Infrastructure, QA, and Production Support teams.
• Participate in production incident management, troubleshooting, and resolution.
• Maintain operational documentation, configuration standards, and runbooks.
• Follow best practices for Kafka availability, reliability, scalability, security, and disaster recovery.

Skills

Kafka administration

Production support

Kafka cluster management

Monitoring

Troubleshooting

Python scripting

Linux/Unix

Shell scripting

Networking

Kafka security

Performance tuning

Capacity planning

Education

Bachelor's degree in CS/IT

Tools

Kafka

Confluent Platform

ZooKeeper / KRaft

Docker

Kubernetes

Prometheus

Grafana

ELK

AWS / Azure / GCP

Python

Git

Linux/Unix shell

Job description

About the Role

We are seeking an experienced Kafka Administrator with strong Python scripting skills to join our engineering team. The ideal candidate will have hands‑on experience in Apache Kafka administration, cluster management, monitoring, troubleshooting, performance tuning, and production support, along with the ability to use Python for automation and operational tooling. The role will involve managing and supporting Kafka environments, ensuring high availability, reliability, scalability, and optimal performance of Kafka clusters.

Key Responsibilities

• Administer, monitor, and maintain Apache Kafka clusters in development, testing, and production environments.
• Manage Kafka brokers, topics, partitions, replication factors, consumer groups, and configurations.
• Monitor Kafka cluster health, broker performance, replication status, consumer lag, throughput, and disk utilization.
• Troubleshoot Kafka issues related to broker failures, under‑replicated partitions, consumer lag, rebalancing, connectivity, performance, and message delivery.
• Perform Kafka performance tuning and capacity planning based on workload and growth requirements.
• Configure and manage Kafka retention, partitioning, replication, and other topic‑level settings.
• Manage Kafka security including authentication, authorization, ACLs, and secure communication.
• Support Kafka cluster installation, configuration, upgrades, patching, and migration activities.
• Implement and maintain monitoring and alerting for Kafka infrastructure.
• Develop Python scripts and automation tools for Kafka administration, monitoring, health checks, reporting, and operational activities.
• Automate routine administrative and support activities using Python and shell scripting.
• Analyze Kafka logs and metrics to identify issues and perform root cause analysis.
• Support Kafka producers and consumers by troubleshooting connectivity, configuration, throughput, and consumer‑group related issues.
• Collaborate with Application Development, DevOps, Infrastructure, QA, and Production Support teams.
• Participate in production incident management, troubleshooting, and resolution.
• Maintain operational documentation, configuration standards, and runbooks.
• Follow best practices for Kafka availability, reliability, scalability, security, and disaster recovery.

Must‑Have Skills

3–5 years of hands‑on experience with Apache Kafka administration and production support.

• Strong understanding of Kafka architecture, including brokers, controllers, topics, partitions, replicas, producers, consumers, and consumer groups.
• Hands‑on experience with Kafka cluster administration and configuration.
• Strong experience troubleshooting Kafka issues in production environments.
• Good understanding of partitioning, replication, ISR, leader election, retention, consumer offsets, and consumer lag.
• Experience with Kafka monitoring, metrics, alerting, and performance troubleshooting.
• Experience managing Kafka topics, partitions, consumer groups, and broker configurations.
• Understanding of Kafka security, ACLs, authentication, authorization, and SSL/TLS.
• Experience with Python scripting for automation and operational tasks.
• Good understanding of Linux/Unix environments and command‑line tools.
• Experience with Shell/Bash scripting.
• Experience with Git and version control practices.
• Strong debugging, troubleshooting, and problem‑solving skills.
• Good understanding of networking concepts relevant to Kafka, including TCP/IP, ports, DNS, and connectivity troubleshooting.

Good‑to‑Have Skills

• Experience with Kafka Connect and/or Kafka Streams.
• Experience with Confluent Kafka / Confluent Platform.
• Experience with ZooKeeper‑based and/or KRaft‑based Kafka environments.
• Experience with monitoring tools such as Prometheus, Grafana, Datadog, Splunk, or ELK.
• Experience with Docker and Kubernetes.
• Experience managing Kafka on AWS, Azure, or GCP.
• Knowledge of cloud‑managed Kafka services such as Amazon MSK, Confluent Cloud, or Azure Event Hubs.
• Understanding of Kafka disaster recovery, backup/restore, replication, and multi‑cluster architectures.
• Experience with CI/CD and infrastructure automation.
• Familiarity with databases such as PostgreSQL, MySQL, MongoDB, or similar technologies.

Preferred Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Experience working in Agile/Scrum environments.
• Strong communication and stakeholder management skills.
• Ability to work effectively with development, infrastructure, DevOps, and production support teams.