Data Engineer

BPMLinks; Hyderabad; On-site; INR 800,000 - 1,200,000; Full time

Job summary

A technology solutions company in Hyderabad is seeking a skilled Data Engineer with experience in AWS Managed Streaming for Apache Kafka (MSK). This role involves designing and maintaining high-performance data pipelines, collaborating with data architects, and ensuring seamless data integration. The ideal candidate should have a degree in computer science or engineering and 3-5 years of experience in a production environment.

Qualifications

• 3-5 years of experience with AWS MSK or Apache Kafka in production.
• Strong understanding of distributed systems.
• Excellent problem-solving skills.

Responsibilities

• Design, develop, and manage real-time data pipelines using AWS MSK.
• Integrate AWS MSK with various data sources.
• Monitor and optimize AWS MSK cluster performance.

Skills

AWS MSK

Real-time data processing

Python

Cloud-based data streaming

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

AWS S3

AWS Glue

AWS Lambda

Docker

Kubernetes

Job description

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from BPMLinks

Employer Branding |Stake Holder Management| Vendor Management| Recruitment|#Top Recruiting Voice Badge |Human Resource Executive| |12K LinkedIn…

Job Summary:

We're looking for a skilled Data Engineer with expertise in AWS Managed Streaming for Apache Kafka (MSK) to design, build, and maintain high-performance data pipelines. You'll work closely with data architects and software engineers to ensure seamless data integration and streaming capabilities using AWS MSK.

Responsibilities:

- Design, develop, and manage real-time data pipelines using AWS MSK

- Integrate AWS MSK with various data sources and consumers

- Monitor and optimize AWS MSK cluster performance and costs

- Collaborate with DevOps to ensure optimal performance and uptime

- Implement security best practices for data streaming and storage

- Document technical processes and best practices

- Work with AWS services like S3, Glue, and Lambda for data processing and integration

Requirements:

- Bachelor’s degree in computer science, Engineering, or a related field

- 3-5 years of experience working with AWS MSK or Apache Kafka in a production environment

- Strong understanding of distributed systems, real-time data processing, and cloud-based data streaming

- Proficiency in programming languages such as Python

- Experience with AWS services like S3, Glue, and Lambda

- Excellent problem-solving and debugging skills

Nice-to-Have Skills:

- Experience with containerization technologies like Docker and Kubernetes

- Familiarity with other messaging systems like RabbitMQ or Apache Pulsar

- Experience with data serialization formats like Avro, Protobuf, or JSON

- Knowledge of Snowflake or other cloud-based data warehousing solutions

Interested candidate can apply or share the updated resume on [e-mail]

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Production

• Industries

  Software Development

Referrals increase your chances of interviewing at BPMLinks by 2x

Sign in to set job alerts for “Data Engineer” roles.

Full Stack Developer SDE 1 ( Java/ Python)

SDE-1 (Full Stack)- Java Software Developer

Full Stack Developer SDE 1 ( Java/ Python)

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.