Senior Kafka Engineer

Solugenix; Hyderabad; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Solugenix is seeking a skilled Confluent Kafka Support Engineer to provide expert guidance on Apache Kafka and Confluent Platform. You will resolve production issues, optimize performance, and ensure reliability of distributed streaming systems.

The role emphasizes troubleshooting, incident management, and postmortem reporting. You will work with Kafka clusters, Confluent components, and cloud-based AWS services, while driving automation through Terraform, Ansible, and scripts.

Qualifications

• 3-5 years of hands-on experience with Apache Kafka and Confluent Kafka.
• Strong knowledge of Kafka internals: brokers, zookeepers, partitions, replication, offsets.
• Experience with Kafka Connect, Schema Registry, REST Proxy, and Kafka security.
• Hands-on AWS experience (EC2, IAM, CloudWatch, S3, Lambda, VPC, Load balancers).
• Scripting and automation using Terraform, Ansible, or equivalent.
• Familiarity with DevOps practices and monitoring tools (Prometheus, Grafana, Splunk, Datadog).
• Experience with Docker and Kubernetes is a plus.

Responsibilities

• Provide L2/L3 technical support for Kafka-based systems and troubleshoot issues.
• Diagnose performance bottlenecks, latency, and data inconsistencies; ensure SLA adherence.
• Maintain and optimize Apache Kafka clusters and Confluent Platform components.
• Assist customers with topic configurations, replication, and fault tolerance; root-ccause analysis.
• Automate operational tasks with Python, shell scripts, Ansible, and Terraform.
• Develop runbooks, SOPs, and knowledge articles; contribute to process improvements.

Skills

Apache Kafka

Confluent Kafka

Kafka internals

AWS

Terraform

Ansible

CI/CD

Monitoring tools

Docker

Kubernetes

Tools

Schema Registry

Kafka Connect

ksqlDB

Control Center

REST Proxy

Prometheus

Grafana

Splunk

Datadog

Docker

Kubernetes

Job description

Job Summary: We are looking for a skilled Confluent Kafka Support Engineer responsible for providing technical support, troubleshooting, and expert guidance for customers using Apache Kafka and the Confluent Platform. The role involves resolving production issues, optimizing system performance, and ensuring the reliability of distributed streaming platforms.

1. Customer Support & Issue Resolution

• Provide L2/L3 technical support for Kafka-based systems.
• Troubleshoot cluster, broker, producer, and consumer issues.
• Diagnose and resolve performance bottlenecks, latency issues, and data inconsistencies.
• Handle incidents and ensure SLA adherence.

2. Kafka & Confluent Platform Expertise

• Support and maintain:
• Apache Kafka clusters
• Confluent Platform components (Schema Registry, Kafka Connect, ksqlDB, Control Center)
• Assist customers with:
• Topic configurations
• Replication and fault tolerance
• Conduct deep-dive investigations into system failures.
• Use logs, metrics, and traces to identify root causes.
• Provide postmortem reports and preventive recommendations.

4. Automation & Process Improvement

• Automate operational tasks using Python, Shell scripts, Ansible, and Terraform.
• Develop runbooks, SOPs, and knowledge articles.
• Continuously improve support processes and efficiency.

Required Skills:

• 3-5 years of experience working with Apache Kafka and Confluent Kafka.
• Strong knowledge of Kafka internals (brokers, zookeepers, partitions, replication, offsets).
• Experience with Kafka Connect, Schema Registry, REST Proxy, and Kafka security.
• Hands-on experience with AWS (EC2, IAM, CloudWatch, S3, Lambda, VPC, Load balancers).
• Proficiency in scripting and automation using Terraform, Ansible, or similar tools.
• Familiarity with DevOps practices and tools (CI/CD pipelines, monitoring tools like Prometheus/Grafana, Splunk, Datadog, etc).
• Experience with containerization (Docker, Kubernetes) is a plus.

Nice to Have:

• Confluent Certified Developer or Administrator.