CHTR - B2 - Senior Kafka Engineer / Administrator - Agentic AI

Capgemini; St. Louis (MO); On-site; USD 45,000 - 70,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

Retirement benefits

Job summary

Tundra Technical Solutions is seeking a senior Kafka Engineer/Administrator to own day-to-day Kafka administration across on-premises and cloud environments, lead the migration to AWS MSK, and coordinate with producer/consumer teams for a smooth cutover.

The role requires hands-on experience with Cloudera, Confluent Platform, Terraform, and GitLab CI/CD, plus strong monitoring via Datadog and robust security practices. This is a contract-based, on-site opportunity in St. Louis, MO.

Qualifications

• 5+ years data/platform/infrastructure engineering experience.
• 4+ years operating Apache Kafka in production.
• Hands-on admin experience with Cloudera and Confluent.
• Experience migrating Kafka workloads to cloud (AWS MSK).
• Terraform and GitLab CI/CD for automation.
• Datadog monitoring, dashboards, and alerting.
• Strong Linux, networking, and scripting skills.

Responsibilities

• Kafka Administration across On-Prem and Cloud (Cloudera/Confluent and AWS MSK).
• Manage topics, partitions, replication, retention, quotas, ACLs, consumer groups.
• Tune brokers, producers, and consumers for throughput and latency.
• Handle patching, upgrades, capacity planning, health/troubleshooting.
• Configure security: TLS/SSL, SASL, IAM for MSK, RBAC/ACLs, encryption.
• Migrate clusters/workloads to AWS MSK; design cutover strategies.
• Build and maintain Kafka/MSK infra with Terraform.
• CI/CD via GitLab; GitOps for topic/ACL/cluster configs.
• Observability: Datadog dashboards, alerts; define SLAs/SLOs.

Skills

Kafka administration

Terraform

GitLab CI/CD

Datadog monitoring

AWS MSK

Cloudera/Confluent

Python Bash scripting

Tools

Terraform

GitLab

Datadog

AWS MSK

Cloudera

Confluent Platform

Kubernetes

Job description

CHTR - B2 - Senior Kafka Engineer / Administrator - Agentic AI (Contract)

St. Louis, MO, United States (On-site)

Contract (1 month 9 days)

Published 2 days ago

Monitoring and observability

technical communication

We are migrating a portfolio of Apache Kafka clusters from on-premises platforms (Cloudera and Confluent) to AWS MSK, managed through a CI/CD pipeline built on Terraform and GitLab.

We are looking for a senior, hands-on Kafka engineer to own day-to-day Kafka administration across both environments, drive the migration, and coordinate closely with producer and consumer application teams to ensure a smooth, low-risk cutover.

This is a senior individual-contributor role for someone who has run Kafka at scale in production, is comfortable in a hybrid on-prem/cloud world, and can act as the technical point of contact between platform and application teams.

Key Responsibilities:

Kafka Administration (On-Prem + Cloud) Administer and operate Kafka clusters across Cloudera, Confluent Platform, and AWS MSK.

Manage topics, partitions, replication, retention, quotas, ACLs, and consumer groups.

Tune brokers, producers, and consumers for throughput, latency, and reliability.

Handle patching, upgrades, capacity planning, and cluster health/performance troubleshooting.

Configure and maintain security: TLS/SSL, SASL, mTLS, IAM (for MSK), RBAC/ACLs, and encryption at rest and in transit.

Migration to AWS MSK Plan and execute migration of clusters and workloads from on-prem (Cloudera/Confluent) to AWS MSK.

Design cutover strategies (e.g., MirrorMaker 2 / Confluent Replicator / cluster linking) with minimal downtime and no data loss. Validate topic parity, offsets, throughput, and data integrity pre- and post-migration.

Define rollback plans and run migration dry-runs in lower environments.

CI/CD & Infrastructure as Code

Build and maintain Kafka/MSK infrastructure using Terraform.

Manage automated deployments and configuration through GitLab CI/CD pipelines.

Implement GitOps practices for topic, ACL, and cluster configuration management.

Enforce version control, code review, and environment promotion (dev test prod).

Observability & Monitoring Build and maintain monitoring, alerting, and dashboards in Datadog for Kafka/MSK (broker metrics, consumer lag, partition health, JVM/GC, throughput).

Establish SLAs/SLOs and proactive alerting to reduce incidents and mean-time-to-resolution.

Correlate Kafka metrics with application and infrastructure telemetry.

Producer / Consumer Coordination Serve as the primary technical liaison between the Kafka platform team and producer/consumer application teams.

Advise application teams on client configuration, serialization/schema management, partitioning strategy, idempotence, and exactly-once/at-least-once semantics.

Support onboarding of new producers/consumers and troubleshoot client-side issues (rebalancing, lag, poison messages).

Coordinate migration schedules, testing, and cutover windows with each impacted team.

Required Qualifications:

5+ years in data/platform/infrastructure engineering, with 4+ years operating Apache Kafka in production.

Hands-on administration experience with Cloudera and Confluent Platform.

Production experience with AWS MSK (provisioning, configuration, IAM auth, networking).

Proven experience migrating Kafka workloads to the cloud with minimal disruption.

Strong Terraform and GitLab CI/CD experience for infrastructure automation.

Hands-on Datadog experience for monitoring, dashboards, and alerting.

Deep understanding of Kafka internals: partitions, replication, ISR, offsets, consumer groups, and delivery semantics.

Solid AWS fundamentals: VPC, security groups, IAM, KMS, CloudWatch, PrivateLink.

Strong Linux, networking, and scripting skills (Bash, Python).

Excellent communication and coordination skills to work across producer/consumer teams.

The pay range that the employer in good faith reasonably expects to pay for this position is $32.36/hour - $50.56/hour. Our benefits include medical, dental, vision and retirement benefits. Applications will be accepted on an ongoing basis.

Tundra Technical Solutions is among North America’s leading providers of Staffing and Consulting Services. Our success and our clients’ success are built on a foundation of service excellence. We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic. Qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law, including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Unincorporated LA County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: client provided property, including hardware (both of which may include data) entrusted to you from theft, loss or damage; return all portable client computer hardware in your possession (including the data contained therein) upon completion of the assignment, and; maintain the confidentiality of client proprietary, confidential, or non-public information. In addition, job duties require access to secure and protected client information technology systems and related data security obligations.