Senior AWS Kafka Engineer (Confluent) - Real-Time Data

Quantum Technologies. LLC; Maryland Heights (MO); Remote; USD 77,000 - 110,000; Full time

Job summary

Quantum Technologies. LLC in Maryland Heights, MO is seeking a Kafka platform engineer to design, deploy, and maintain highly available Kafka and Confluent Platform clusters for real-time data processing across production and non-production environments.

The role emphasizes security, automation with Terraform/Ansible, CI/CD, incident response, and collaboration with application teams to optimize reliability, performance, and cost while ensuring governance and disaster recovery readiness.

Qualifications

• 3+ years of hands-on experience administering enterprise Kafka or Confluent Platform environments.
• 3+ years troubleshooting Kafka issues including broker availability, replication, partitioning, lag, throughput, and Connect task failures.
• 3+ years automating infrastructure with Terraform and Ansible, with Git-based CI/CD pipelines and Python or scripting.
• Experience designing highly available Kafka architectures with cross-cluster replication, failover, backup, and DR testing.
• Deep understanding of Kafka security, including TLS, SASL, ACLs, RBAC, Schema Registry compatibility, and credentials management.

Responsibilities

• Design, deploy, configure, upgrade, and maintain Kafka/Confluent clusters for real-time processing across environments.
• Monitor platform health, broker performance, replication, partition balance, throughput, storage, and consumer lag; troubleshoot incidents for reliability.
• Build and support Kafka topics, partitions, replication policies, schemas, and Connect integrations (S3, Trino, OpenSearch); assist application/data teams.

Skills

Client-facing communication

English proficiency

Tools

Terraform

Ansible

Python scripting

Prometheus

Grafana

OpenSearch

Job description

Quantum Technologies. LLC in Maryland Heights, MO is seeking a Kafka platform engineer to design, deploy, and maintain highly available Kafka and Confluent Platform clusters for real-time data processing across production and non-production environments.

The role emphasizes security, automation with Terraform/Ansible, CI/CD, incident response, and collaboration with application teams to optimize reliability, performance, and cost while ensuring governance and disaster recovery readiness.