Kafka Admin

BPMLinks; Hyderabad; Presencial; INR 900 000 - 1 300 000; Tempo integral

Resumo da oferta

BPMLinks in Hyderabad, India seeks a skilled Kafka Admin with AWS MSK expertise to design and maintain high-performance data pipelines and ensure seamless data streaming across systems.

You will work with data architects and software engineers, implement security best practices, optimize costs, and integrate with AWS services such as S3, Glue, and Lambda, using Python for automation.

Qualificações

• 4–5 years of experience with AWS MSK or Apache Kafka in production.
• Strong knowledge of distributed systems and real-time data processing.
• Proficiency in Python and AWS services (S3, Glue, Lambda).

Responsabilidades

• Design, develop, and manage real-time data pipelines using AWS MSK
• Integrate AWS MSK with various data sources and consumers
• Monitor and optimize AWS MSK cluster performance and costs
• Collaborate with DevOps to ensure optimal performance and uptime
• Implement security best practices for data streaming and storage
• Document technical processes and best practices
• Work with AWS services like S3, Glue, and Lambda for data processing and integration

Conhecimentos

AWS MSK

Apache Kafka

Python

Distributed systems

Real-time processing

Docker

Kubernetes

Formação académica

Bachelor’s degree in computer science or engineering

Ferramentas

Docker

Kubernetes

Descrição da oferta de emprego

Job Summary:

We're looking for a skilled Kafka Admin with expertise in AWS Managed Streaming for Apache Kafka (MSK) to design, build, and maintain high-performance data pipelines. You'll work closely with data architects and software engineers to ensure seamless data integration and streaming capabilities using AWS MSK.

Responsibilities:

• Design, develop, and manage real-time data pipelines using AWS MSK
• Integrate AWS MSK with various data sources and consumers
• Monitor and optimize AWS MSK cluster performance and costs
• Collaborate with DevOps to ensure optimal performance and uptime
• Implement security best practices for data streaming and storage
• Document technical processes and best practices
• Work with AWS services like S3, Glue, and Lambda for data processing and integration

Requirements:

• Bachelor’s degree in computer science, Engineering, or a related field
• 4-5 years of experience working with AWS MSK or Apache Kafka in a production environment
• Strong understanding of distributed systems, real-time data processing, and cloud-based data streaming
• Proficiency in programming languages such as Python
• Experience with AWS services like S3, Glue, and Lambda
• Excellent problem-solving and debugging skills

Nice-to-Have Skills:

• Experience with containerization technologies like Docker and Kubernetes
• Familiarity with other messaging systems like RabbitMQ or Apache Pulsar
• Experience with data serialization formats like Avro, Protobuf, or JSON
• Knowledge of Snowflake or other cloud-based data warehousing solutions