Kafka to MSK Migration Engineer

Insight Global; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

Insight Global seeks a Kafka/MSK migration specialist to assess on-prem Kafka workloads and plan migration to AWS MSK, including topic/partition layouts, consumer groups, latency requirements, and security considerations.

You will prototype and execute UAT and production parallel testing with MirrorMaker 2, optimize MSK clusters, coordinate with application, infrastructure, and platform teams, and oversee cutover, validation, and post-release activities while documenting best practices.

Qualifications

• Hands-on experience with AWS services and migration patterns.
• Deep Kafka knowledge including brokers, topics, partitions, and consumer groups.
• Experience with AWS MSK provisioning, monitoring, and security.
• Familiarity with MirrorMaker 2 or similar tooling.
• Experience with cloud migration, testing, cutover, and validation.
• Knowledge of Kafka ecosystem components (Schema Registry, Kafka Connect, Kafka Streams).
• Understanding of VPC, TLS, IAM, and hybrid networking.
• Programming/scripting skills in Python/Java/Shell to automate tasks.
• Familiarity with RESTful APIs, API Gateway patterns.
• CI/CD and version control experience (GitLab, Jenkins, Maven).
• Observability tooling (CloudWatch, Prometheus, Grafana).
• Ability to design repeatable automation and AI-assisted approaches.

Responsibilities

• Assess on-prem Kafka clusters and determine migration readiness.
• Plan and provision MSK infrastructure, sizing, storage, networking.
• Execute UAT and production parallel testing with MirrorMaker 2.
• Tune MSK performance for stability and scalability.
• Coordinate migration cutover with development, infra, and platforms.
• Oversee post-release validation and rollback considerations.
• Decommission legacy on-prem clusters after migration.
• Document procedures and share best practices.

Skills

AWS Services

Apache Kafka

AWS MSK

Migration Tooling

Cloud Migration

Kafka Ecosystem

Networking & Security

Programming & Automation

APIs & Integration

CI/CD

Observability

Automation Excellence

Tools

MirrorMaker 2

Confluent Replicator

Kafka Connect

Job description

Insight Global seeks a Kafka/MSK migration specialist to assess on-prem Kafka workloads and plan migration to AWS MSK, including topic/partition layouts, consumer groups, latency requirements, and security considerations.

You will prototype and execute UAT and production parallel testing with MirrorMaker 2, optimize MSK clusters, coordinate with application, infrastructure, and platform teams, and oversee cutover, validation, and post-release activities while documenting best practices.