Kafka Admin

BPMLinks; Hyderabad; On-site; INR 1,500,000 - 2,400,000; Full time

Job summary

BPMLinks in India is seeking a Kafka Admin with AWS MSK expertise to design, build, and maintain high-performance data pipelines. You will collaborate with data architects and software engineers to ensure seamless data integration and streaming capabilities.

The role requires 4–5 years of production experience with AWS MSK or Kafka, strong knowledge of distributed systems, and Python proficiency. You will work closely with DevOps and leverage S3, Glue, and Lambda to deliver reliable data

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 4-5 years of experience working with AWS MSK or Apache Kafka in production.
• Strong understanding of distributed systems and real-time data processing.
• Proficiency in Python.
• Experience with AWS services like S3, Glue, and Lambda.

Responsibilities

• Design, develop, and manage real-time data pipelines using AWS MSK.
• Integrate AWS MSK with various data sources and consumers.
• Monitor and optimize AWS MSK cluster performance and costs.
• Collaborate with DevOps to ensure optimal performance and uptime.
• Implement security best practices for data streaming and storage.
• Document technical processes and best practices.
• Work with AWS services like S3, Glue, and Lambda for data processing and integration.

Skills

Kafka Admin

AWS MSK

Python

Distributed systems

Real-time data processing

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS S3

AWS Glue

AWS Lambda

Docker

Kubernetes

Apache Kafka tooling

Snowflake

Protobuf/Avro/JSON

RabbitMQ

Apache Pulsar

Job description

Job Summary:

We're looking for a skilled Kafka Admin with expertise in AWS Managed Streaming for Apache Kafka (MSK) to design, build, and maintain high-performance data pipelines. You'll work closely with data architects and software engineers to ensure seamless data integration and streaming capabilities using AWS MSK.

Responsibilities:

• Design, develop, and manage real-time data pipelines using AWS MSK
• Integrate AWS MSK with various data sources and consumers
• Monitor and optimize AWS MSK cluster performance and costs
• Collaborate with DevOps to ensure optimal performance and uptime
• Implement security best practices for data streaming and storage
• Document technical processes and best practices
• Work with AWS services like S3, Glue, and Lambda for data processing and integration

Requirements:

• Bachelor’s degree in computer science, Engineering, or a related field
• 4-5 years of experience working with AWS MSK or Apache Kafka in a production environment
• Strong understanding of distributed systems, real-time data processing, and cloud-based data streaming
• Proficiency in programming languages such as Python
• Experience with AWS services like S3, Glue, and Lambda
• Excellent problem-solving and debugging skills

Nice-to-Have Skills:

• Experience with containerization technologies like Docker and Kubernetes
• Familiarity with other messaging systems like RabbitMQ or Apache Pulsar
• Experience with data serialization formats like Avro, Protobuf, or JSON
• Knowledge of Snowflake or other cloud-based data warehousing solutions