Senior Kafka Engineer: Cloud MSK Migration & Automation

Droisys; Charlotte (NC); Hybrid; USD 83,000 - 90,000; Full time

Job summary

Droisys is seeking a senior Kafka Admin to own day-to-day administration across on-prem and cloud environments, leading a critical migration to AWS MSK. This senior IC role requires hands-on experience with large Kafka deployments, Terraform-based infrastructure, and CI/CD pipelines in a hybrid cloud setting.

You will coordinate with producer/consumer teams, ensure data integrity, and implement robust cutover strategies with minimal downtime.

Qualifications

• 5+ years in data/platform/infrastructure engineering.
• 4+ years operating Apache Kafka in production.
• Experience with Cloudera and Confluent Platform.
• Proven experience migrating Kafka workloads to the cloud.
• Terraform and GitLab CI/CD for infrastructure automation.

Responsibilities

• Kafka Administration (On-Prem + Cloud).
• Administer and operate Kafka clusters across Cloudera, Confluent Platform, and AWS MSK.
• Manage topics, partitions, replication, retention, quotas, ACLs, and consumer groups.
• Tune brokers, producers, and consumers for throughput, latency, and reliability.
• Handle patching, upgrades, capacity planning, and cluster health troubleshooting.
• Configure and maintain security: TLS/SSL, SASL, mTLS, IAM, RBAC/ACLs, encryption at rest/in transit.
• Plan and execute migration to AWS MSK; design cutover strategies with minimal downtime/data loss.
• CI/CD & Infra as Code: Terraform; GitLab CI/CD pipelines; GitOps for config mgmt.
• Observability: Datadog dashboards, alerts, and monitoring of broker metrics and lag.
• Coordinate migration schedules with impacted teams; act as liaison between platform and apps.

Skills

Kafka administration

Cloud migration

Team coordination

Linux scripting

Education

Bachelor's degree in CS or related field

Tools

Terraform

GitLab CI/CD

Datadog

AWS MSK

Confluent Platform

Job description

Droisys is seeking a senior Kafka Admin to own day-to-day administration across on-prem and cloud environments, leading a critical migration to AWS MSK. This senior IC role requires hands-on experience with large Kafka deployments, Terraform-based infrastructure, and CI/CD pipelines in a hybrid cloud setting.

You will coordinate with producer/consumer teams, ensure data integrity, and implement robust cutover strategies with minimal downtime.