Professional MW Messaging Infrastructure Engineer - Kafka

LPL Financial Global Capability Center; Hyderabad; On-site; INR 4,200,000 - 6,600,000; Full time

Job summary

LPL Financial Global Capability Center in Hyderabad seeks a Professional Middleware Messaging Infrastructure Engineer focused on Apache Kafka to deploy, manage, and scale streaming platforms. You’ll ensure high-throughput, real-time data pipelines for critical applications, collaborating with diverse teams across the enterprise.

Ideal candidates bring 6+ years in middleware with strong Kafka, Confluent, and cloud experience, and a passion for reliability, security, and continuous improvement in

Qualifications

• 6+ years middleware experience with 3+ years in Apache Kafka.
• Deep expertise in Kafka cluster management, replication and failover.
• Experience with Kafka Connect for data integration.
• Familiarity with Kafka Streams or KSQL for real-time processing.
• Unix/Linux administration and Python/Shell scripting.
• Experience with Confluent Platform, AWS MSK, or Azure Event Hubs.
• Bachelor’s in CS/IT or related field; Confluent certification preferred.

Responsibilities

• Install, configure, and manage Kafka clusters, brokers, ZooKeeper/KRaft, and Confluent components.
• Design Kafka topic architecture, partitioning, and replication policies.
• Monitor health metrics, consumer lag, and throughput with Prometheus/Grafana.
• Troubleshoot consumer groups, offsets, and message replay scenarios.
• Implement Kafka security: SSL/TLS, SASL, and ACL management.
• Collaborate with dev teams to build event-driven solutions.
• Automate provisioning, topic management, and configurations via scripts and IaC.
• Plan capacity, benchmark performance, and tune Kafka clusters.

Skills

Kafka

Kafka Connect

Kafka Streams

KSQL

Unix/Linux

Python

Shell scripting

Confluent Platform

AWS MSK

Azure Event Hubs

CAP theorem

Education

Bachelor's degree in Computer Science/IT

Tools

Confluent Platform

Kubernetes

Schema Registry/Avro

Prometheus

Grafana

ZooKeeper/KRaft

Terraform/ IaC

Job description

Job Description:

Where Ambition Meets Innovation

At LPL’s Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.

Join us as we create something extraordinary together.

Job Overview

The Professional Middleware Messaging Infrastructure Engineer specializing in Apache Kafka is responsible for deploying, managing, and scaling Kafka-based event streaming platforms across the enterprise. This role ensures reliable, high-throughput data pipelines and real-time messaging for critical business applications.

Responsibilities

• Install, configure, and manage Apache Kafka clusters including brokers, ZooKeeper/KRaft, and Confluent components
• Design and implement Kafka topic architecture, partitioning strategies, and replication policies
• Monitor Kafka health metrics, consumer lag, and throughput using tools such as Prometheus and Grafana
• Troubleshoot consumer group issues, offset management, and message replay scenarios
• Implement Kafka security including SSL/TLS, SASL authentication, and ACL management
• Collaborate with development teams to design Kafka-based event-driven solutions
• Automate Kafka provisioning, topic management, and configuration using scripts and IaC tools
• Conduct capacity planning, performance benchmarking, and tuning for Kafka clusters

What are we looking for?

We’re looking for strong collaborators who deliver exceptional client experiences and thrive in fast-paced, team-oriented environments. Our ideal candidates pursue greatness, act with integrity, and are driven to help our clients succeed. We value those who embrace creativity, continuous improvement, and contribute to a culture where we win together and create and share joy in our work.

Requirements

• 6+ years of middleware experience with 3+ years focused on Apache KafkaDeep expertise in Kafka cluster management, replication, and failover
• Experience with Kafka Connect for data integration with databases and external systems
• Familiarity with Kafka Streams or KSQL for real-time data processing
• Strong Unix/Linux administration skills and Python/Shell scripting
• Experience with Confluent Platform, AWS MSK, or Azure Event Hubs
• Understanding of distributed systems concepts and CAP theorem
• Bachelor's degree in Computer Science, Information Technology, or related field
• Confluent Certified Administrator or Developer certification preferred

Preference

• Experience with schema management using Avro and Confluent Schema Registry
• Knowledge of Kubernetes and containerized Kafka deployments
• Exposure to Apache Flink or Spark Streaming for advanced stream processing
• EDUCATION & QUALIFICATIONS

LPL Global Business Services, LLP - PRIVACY POLICY

Requirements: