Senior AWS Kafka Engineer - Remote

Machine Learning Technologies, LLC; Maryland Heights (MO); Remoto; USD 77.000 - 110.000; Tempo pieno

Descrizione del lavoro

Machine Learning Technologies, LLC is seeking a Kafka Platform Engineer to design, deploy, and operate scalable Kafka environments for real-time data processing. You will monitor health, performance, and security, and implement Connect integrations with S3, Trino, and OpenSearch.

The role requires strong scripting, automation, and DR planning. Ideal candidates have hands-on Kafka/Confluent experience, Terraform and Ansible skills, and proven ability to lead cross-team incident resolution in

Competenze

• 3+ years administering enterprise Apache Kafka/Confluent Platform environments, including production cluster design and upgrades.
• 3+ years troubleshooting Kafka issues including broker availability, replication, rebalance, lag, and Connect failures.
• 3+ years automating infra with Terraform/Ansible, with Git-based CI/CD and Python or scripting.
• Experience designing highly available Kafka architectures with cross-cluster replication and disaster recovery.
• Deep understanding of Kafka security: TLS, SASL, ACLs, RBAC, and audit controls.
• Experience designing observability using Prometheus, Grafana, OpenSearch, or similar tooling.
• Ability to define engineering standards and guide producers/consumers.
• Strong client-facing communication and incident leadership in English.
• Office presence, on-call rotation, collaboration with vendor support.

Mansioni

• Design, deploy, and maintain scalable Kafka clusters for real-time data processing.
• Monitor broker health, throughput, lag, and storage; tune for reliability and performance.
• Build topics, replication policies, schemas, and Connect integrations (S3, Trino, OpenSearch).
• Implement security controls: TLS, SASL, ACLs, and credential management.
• Automate provisioning, configuration, scaling, upgrades, and patching using Terraform, Ansible, and Python.
• Plan backup/recovery, cross-cluster replication, and DR testing for defined SLAs.
• Participate in on-call rotation, lead troubleshooting, and coordinate with teams and vendors.
• Provide Kafka architecture guidance and maintain runbooks and capacity plans.

Conoscenze

Apache Kafka

Confluent Platform

Terraform

Ansible

Python scripting

CI/CD

Git workflows

Kafka Connect

OpenSearch

Monitoring dashboards

Strumenti

AWS

Kubernetes

CI/CD tools

Descrizione del lavoro

Machine Learning Technologies, LLC is seeking a Kafka Platform Engineer to design, deploy, and operate scalable Kafka environments for real-time data processing. You will monitor health, performance, and security, and implement Connect integrations with S3, Trino, and OpenSearch.

The role requires strong scripting, automation, and DR planning. Ideal candidates have hands-on Kafka/Confluent experience, Terraform and Ansible skills, and proven ability to lead cross-team incident resolution in