Kafka To AWS MSK Migration Engineer

Tech Mahindra; Bengaluru; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Tech Mahindra seeks an experienced Kafka Migration Engineer to lead on-prem Kafka to AWS MSK migrations. The role covers cluster discovery, capacity planning, testing, cutover, and decommissioning with automation and AI-assisted tooling.

Strong Java and Unix/Linux fundamentals are required, with 5–10 years of distributed messaging experience. The position focuses on collaboration across application, infrastructure, and platform teams to ensure safe, scalable migrations and production readiness

Qualifications

• 5–10 years of hands-on engineering experience with distributed messaging systems.
• Proficiency in Java or Python and Unix/Linux troubleshooting abilities.
• Familiarity with SDLC, CI/CD, Kubernetes, and cloud migration patterns.

Responsibilities

• Assess on-prem Kafka clusters and plan migration to AWS MSK with topic/partition mapping.
• Configure MSK with proper capacity, security, and networking settings.
• Execute UAT and production cutover, validate results, and troubleshoot issues.
• Automate repeatable migration tasks and contribute to AI-assisted tooling.
• Decommission legacy Kafka clusters after successful migration, with documentation.

Skills

Java

AWS

Kafka

Unix/Linux

Education

Bachelor's or Master's degree in Computer Science or related field

Tools

MirrorMaker 2

Confluent Replicator

Kubernetes

REST APIs

GitLab/Jenkins

Maven

Job description

Kafka to AWS MSK Migration Engineer

Mandatory Skills JAVA/AWS/KAFKA/Amazon AWS MSK and worked Apache Kafka to AWS MSK migration project and strong in Unix/Linux fundamentals.

Location Bangalore

Experience: 9 years to 13 years

Education: Bachelor's or Master's degree in Computer Science is mandatory and any 3 years degree can’t be considered

Note - Need immediate joiners’ profiles only candidate need to join below 15days please apply accordingly.

ETO Factory – Kafka to AWS MSK Migration Engineer

We are seeking a highly skilled Kafka Migration Engineer to join the ETO Factory team and support a high-visibility on-prem Apache Kafka to AWS MSK migration project. This role will focus on cluster discovery, topic mapping, capacity planning, platform configuration, migration execution, validation, performance tuning, automation, and legacy cluster decommissioning. The ideal candidate will bring strong hands-on engineering experience with distributed messaging systems, a

structured delivery mindset, and the ability to collaborate effectively with application, infrastructure, and platform teams to execute migrations safely and at scale.

Key Responsibilities

• Assess on-prem Kafka clusters and perform discovery across topics, partitions, consumer groups, throughput patterns, retention policies, and inter-service dependencies to determine migration readiness.
• Analyze workloads based on message volume, business criticality, ordering guarantees, latency requirements, and consumer group complexity to determine appropriate MSK cluster placement and configuration.
• Plan and provision required AWS MSK infrastructure capacity including broker sizing, storage, networking, and security configurations to support migration, testing, and production execution.
• Execute UAT and production parallel testing using MirrorMaker 2 or similar replication tooling, compare message delivery outcomes, capture evidence, and troubleshoot discrepancies through closure.
• Perform performance tuning and optimization of MSK clusters to ensure migrated workloads are stable, scalable, efficient, and production ready.
• Partner with application engineering, infrastructure, and platform teams to finalize migration plans, consumer/producer cutover approach, validation criteria, and rollback considerations.
• Manage release readiness, execute production cutover activities including consumer group migration, offset synchronization, and complete post-release checkout and validation procedures.
• Support legacy on-premises Kafka cluster decommissioning after successful migration and validation.
• Design, enhance, and implement automation frameworks and AI-assisted solutions to enable repeatable, efficient, large-scale Kafka-to-MSK migrations.
• Document migration procedures, operational learnings, risks, and best practices to improve the factory execution model.

Required Qualifications Basic Qualifications

Education:

Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related quantitative discipline.

Experience:

5 years to 10 years of hands-on engineering experience in a collaborative, team-based environment with distributed messaging systems.

Programming:

Professional proficiency in Java or Python a similar programming/scripting language.

Systems:

Strong Unix/Linux fundamentals with the ability to troubleshoot application, deployment, environment, and runtime issues.

Methodology:

Familiarity with SDLC practices, CI/CD delivery models, change management, and Kubernetes-based deployments.

Technical Competencies

Candidates are not expected to be experts in every tool, but should bring strong hands-on experience across several of the following areas:

AWS Services:

Hands-on experience with AWS compute and migration patterns, including ECS, EKS, Lambda, CloudWatch, and related cloud services.

Apache Kafka:

Deep hands-on experience with Kafka architecture including brokers, topics, partitions, consumer groups, replication, and cluster operations.

AWS MSK:

Practical experience with Amazon Managed Streaming for Apache Kafka including cluster provisioning, configuration, monitoring, and security (IAM, mTLS, SASL/SCRAM).

Migration Tooling:

Experience with MirrorMaker 2, Confluent Replicator, or similar crosscluster replication tools for data migration and offset synchronization.

Cloud Migration:

Practical understanding of on-prem to cloud migration strategies, workload assessment, testing, cutover, and post-migration validation.

Kafka Ecosystem:

Familiarity with Schema Registry, Kafka Connect, Kafka Streams, and related ecosystem components.

Networking and Security:

Understanding of VPC peering, PrivateLink, TLS encryption, IAM policies, and network connectivity patterns for hybrid Kafka architectures.

Programming and Automation:

Experience with Python, Java, Shell, or similar languages to build scripts, utilities, and automation frameworks for topic creation, ACL migration, and consumer group management.

APIs and Integration:

Working knowledge of RESTful APIs, API design, and API Gateway patterns.

CI/CD and Version Control:

Strong experience with GitLab, Jenkins, Maven, or similar tools supporting automated build, test, and deployment pipelines.

Observability:

Familiarity with monitoring, logging, alerting, and troubleshooting using tools such as CloudWatch, Prometheus, Grafana, Splunk, or equivalent platforms for Kafka/MSK cluster health.

Automation Excellence:

Ability to design repeatable automation and apply AI-assisted engineering approaches to improve migration scale, quality, and efficiency.

Core Competencies

Exceptional analytical, troubleshooting, and debugging skills.

Strong ownership mindset with the ability to drive work to closure and meet commitments.

Clear written and verbal communication, including concise status updates, structured briefings, and proactive stakeholder management.

Effective collaboration across application, infrastructure, platform, and global engineering teams.

Ability to work constructively across time zones, build alignment, and resolve issues with urgency and professionalism.

Strong integrity, sound judgment, and commitment to good conduct and ethical decisionmaking.

High energy, intellectual curiosity, and willingness to surface risks early, ask thoughtful questions, and continuously improve.