Kafka Administrator

Purview Services; Gurugram District; On-site; INR 3,000,000 - 5,000,000; Full time

Job summary

Purview Services is seeking a seasoned Kafka and observability engineer to design, build, and operate streaming platforms for logs, metrics, and traces. The role emphasizes robust security, scalable Kafka clusters, and end-to-end observability with OpenTelemetry pipelines.

The ideal candidate brings 5+ years of hands-on Kafka production experience, strong architecture knowledge, and proven ability to troubleshoot complex production issues across telemetry systems.

Qualifications

• 5+ years of hands-on experience with Apache Kafka in production environments.
• Strong knowledge of Kafka architecture: brokers, partitions, replication, consumer groups, offsets.
• Hands-on experience with Kafka security (SSL/TLS, SASL).
• Strong hands-on experience with OpenTelemetry pipelines and collectors.
• Experience integrating logs, metrics, and traces into a unified observability platform.
• Proven expertise in performance tuning and production troubleshooting.
• Experience managing enterprise-scale data or observability platforms.

Responsibilities

• Design, build, and operate Kafka-based streaming platforms for logs, metrics, and traces.
• Configure and optimize Kafka clusters including brokers, topics, partitions, replication, and consumer groups.
• Implement and manage Kafka security using SSL/TLS and SASL mechanisms.
• Design and operate OpenTelemetry pipelines and collectors for metrics, logs, and distributed tracing.
• Integrate telemetry data across logs, metrics, and traces for end-to-end observability.
• Perform performance tuning, capacity planning, and reliability improvements for Kafka and OTel pipelines.
• Troubleshoot complex production issues across Kafka, OpenTelemetry collectors, and downstream systems.
• Manage and operate data/telemetry platforms ensuring stability, scalability, and compliance.
• Collaborate with platform, SRE, and application teams to support enterprise observability use cases.

Skills

Kafka in production

Kafka architecture

Kafka security

OpenTelemetry pipelines

Logs, metrics, traces integration

Performance tuning

Production troubleshooting

Enterprise-scale data platforms

Job description

Role & responsibilities

Key Responsibilities

Design, build, and operate Kafka-based streaming platforms for logs, metrics, and traces.

Configure and optimize Kafka clusters including brokers, topics, partitions, replication, and consumer groups.

Implement and manage Kafka security using SSL/TLS and SASL mechanisms.

Design and operate OpenTelemetry pipelines and collectors for metrics, logs, and distributed tracing.

Integrate telemetry data across logs, metrics, and traces for end-to-end observability.

Perform performance tuning, capacity planning, and reliability improvements for Kafka and OTel pipelines.

Troubleshoot complex production issues across Kafka, OpenTelemetry collectors, and downstream systems.

Manage and operate data/telemetry platforms ensuring stability, scalability, and compliance.

Collaborate with platform, SRE, and application teams to support enterprise observability use cases.

Mandatory Skills (Must-Have)

5+ years of hands-on experience with Apache Kafka in production environments.

Strong knowledge of Kafka architecture: brokers, partitions, replication, consumer groups, offsets.

Hands-on experience with Kafka security (SSL/TLS, SASL).

Strong hands-on experience with OpenTelemetry pipelines and collectors.

Experience integrating logs, metrics, and traces into a unified observability platform.

Proven expertise in performance tuning and production troubleshooting.

Experience managing enterprise-scale data or observability platforms.