AWS Kafka Engineer

ZealHire Inc.; Maryland Heights (MO); On-site; USD 120,000 - 180,000; Full time

Job summary

ZealHire Inc. is seeking an onsite Apache Kafka/Confluent Platform engineer to design, deploy, and maintain scalable real-time data pipelines near Englewood, Colorado or Maryland Heights, Missouri.

The role requires hands-on Kafka expertise, security governance, and experience in automation using Terraform, Ansible, and Python. The successful candidate will monitor platform health, optimize performance, and support cross-team data initiatives while participating in on-call rotation and DR

Qualifications

• 3+ years of hands-on experience administering enterprise Apache Kafka or Confluent Platform environments.
• 3+ years of experience troubleshooting complex Kafka issues.
• 3+ years of experience automating infrastructure and platform operations with Terraform and Ansible, supported by Git-based CI/CD pipelines and Python or shell scripting.
• Experience designing and operating highly available Kafka architectures, including cross-cluster replication, failover, backup, recovery, and disaster recovery testing.
• Deep understanding of Kafka security and governance, including TLS, SASL, ACLs, RBAC, Schema Registry compatibility, service accounts, credential management, and audit controls.
• Experience designing and operating observability capabilities using Prometheus, Grafana, Confluent Control Center, OpenSearch, or equivalent solutions.
• Proven ability to define engineering standards, advise application teams on producer and consumer patterns, review platform designs, and lead technical resolution across multiple teams.
• Proven excellence in client-facing communication, technical documentation, incident leadership, root cause analysis, and consulting skills in English.

Responsibilities

• Design, deploy, configure, upgrade, and maintain Apache Kafka and Confluent Platform clusters for real-time data processing.
• Monitor broker health, replication, partition balance, throughput, and consumer lag; troubleshoot incidents and optimize performance.
• Build and support Kafka topics, partitions, replication policies, schemas, and Kafka Connect integrations (S3, Trino, OpenSearch).
• Maintain platform security and compliance controls (TLS, SASL, ACLs, RBAC, secrets management).
• Automate clustering tasks using Terraform, Ansible, CI/CD pipelines, and Python or shell scripting.
• Test backup, recovery, failover, cross-cluster replication, and DR procedures to meet SLAs and continuity goals.
• Participate in on-call rotation; document root cause analyses and coordinate with application teams, vendors, and cloud support.
• Provide guidance on Kafka architecture and runbooks; stay current with Apache Kafka and Confluent capabilities.

Skills

Communication

Technical documentation

Incident leadership

Root cause analysis

Consulting skills

Tools

Terraform

Ansible

CI/CD pipelines

Python

Shell scripting

Prometheus

Grafana

OpenSearch

Kafka

Confluent Platform

KRaft

ZooKeeper

Schema Registry

AWS

Kubernetes

OpenShift

Job description

ZealHire. Maryland Heights, Missouri, United States Contract Deadline: Nov 19, 2026

USC/GC/GC-EAD/H4-EADNeed LinkedIn, VISA DL

This is an onsite position near Fiddler’s Green Circle, Englewood, Colorado OR Riverport Dr in Maryland Heights, Missouri

LOCAL CANDIDATES OR WITHIN 100MILES

Key Responsibilities

• Design, deploy, configure, upgrade, and maintain highly available, scalable Apache Kafka and Confluent Platform clusters for real-time data processing across production and non-production environments.
• Monitor platform health, broker performance, replication, partition balance, throughput, storage utilization, consumer lag, and connector status; troubleshoot incidents and tune the platform for reliability, performance, and minimal downtime.
• Build and support Kafka topics, partitions, replication policies, schemas, service accounts, and Kafka Connect integrations, including connectors for Amazon S3, Trino, and OpenSearch; assist application and data teams with producer and consumer integration.
• Implement and maintain platform security and compliance controls, including TLS encryption, SASL authentication, access control lists, role-based access control, secrets management, auditability, and secure certificate lifecycle practices.
• Automate cluster provisioning, configuration, scaling, upgrades, patching, and operational tasks using Terraform, Ansible, CI/CD pipelines, and Python or shell scripting.
• Plan and test backup, recovery, failover, cross-cluster replication, and disaster recovery procedures to support defined service-level objectives and business continuity requirements.
• Participate in production support and on-call rotation, lead technical troubleshooting, document root cause analyses, and coordinate resolution with application teams, infrastructure teams, incident managers, vendors, and cloud support personnel.
• Provide technical guidance on Kafka architecture and operational best practices; maintain runbooks, standards, diagrams, capacity plans, and support documentation; and remain current with Apache Kafka and Confluent capabilities.

Required Qualifications

• 3+ years of hands‑on experience administering enterprise Apache Kafka or Confluent Platform environments, including production cluster design, capacity planning, upgrades, patching, performance tuning, and lifecycle management.
• 3+ years of experience troubleshooting complex Kafka issues involving broker availability, replication, partitioning, rebalance behavior, consumer lag, throughput, latency, storage, and Kafka Connect task failures.
• 3+ years of experience automating infrastructure and platform operations with Terraform and Ansible, supported by Git-based CI/CD pipelines and Python or shell scripting.
• Demonstrated experience designing and supporting highly available Kafka architectures, including cross-cluster replication, failover, backup, recovery, and disaster recovery testing.
• Deep understanding of Kafka security and governance, including TLS, SASL, ACLs, RBAC, Schema Registry compatibility, service accounts, credential management, and audit controls.
• Experience designing and operating observability capabilities using Prometheus, Grafana, Confluent Control Center, OpenSearch, or equivalent solutions, including actionable dashboards and alert thresholds.
• Proven ability to define engineering standards, advise application teams on producer and consumer patterns, review platform designs, and lead technical resolution across multiple teams.
• Proven excellence in client‑facing communication, technical documentation, incident leadership, root cause analysis, and consulting skills in English.
• Ability to work in office, participate in an on‑call rotation, and collaborate with senior client resources and vendor support teams.

Preferred Qualifications

• Experience with Confluent Platform or Confluent Cloud components such as KRaft or ZooKeeper, Schema Registry, Kafka Connect, Control Center, Cluster Linking, and stream-processing capabilities.
• Experience operating Kafka on AWS, Kubernetes, Red Hat OpenShift, or another cloud or container platform; AWS Managed Streaming for Apache Kafka (Amazon MSK) experience is beneficial.
• Experience with GitLab, GitHub Actions, Jenkins, Argo CD, or comparable CI/CD and GitOps tooling.
• Experience integrating Kafka with Amazon S3, Trino, OpenSearch, relational databases, IBM MQ, data warehouses, or other enterprise source and sink systems.
• Familiarity with data governance, schema evolution, topic naming and retention standards, capacity forecasting, service-level objectives, and platform cost optimization.
• Relevant certification or formal training in Apache Kafka, Confluent, AWS, Kubernetes, Terraform, Ansible, or related technologies.

Maryland Heights, Missouri, United States 63043 +1 more location

Salary Range:

$000,000 – $000,000 Salary range is not shown on the candidate portal

Maryland Heights, Missouri, United States