Kafka Platform Engineer — Real-Time Data Backbone

aep; Columbus (OH); On-site; USD 99,000 - 129,000; Full time

Job summary

AEP is seeking a Senior Kafka Engineer to design, deploy, and operate the enterprise Kafka platform that forms the data movement backbone. The role focuses on stability, scalability, data safety, and enabling producer/consumer teams.

You will containerize services on OpenShift, implement CI/CD with GitHub Actions and Argo CD, perform Kafka administration, and collaborate with architects and business teams to deliver scalable, cloud-integrated data pipelines.

Qualifications

• Strong hands-on Kafka experience in production environments.
• Experience with Kafka deployments in Kubernetes/OpenShift platforms.
• Knowledge of multi-region HA and data movement patterns.
• Proficiency in Python/Java for microservices and connectors.
• Familiarity with TLS, RBAC, and Kafka security concepts.

Responsibilities

• Deploy and manage Kafka connectors (CDC, JDBC, IBM MQ, Redshift, etc.).
• Containerize and deploy microservices and connectors on OpenShift using Docker, YAML, and Kustomize.
• Implement CI/CD pipelines using GitHub Actions and Argo CD for automated deployments.
• Perform basic Kafka administration: topics, partitions, replication, monitoring.
• Collaborate with architects and business teams to deliver scalable data integrations.
• Monitor Kafka clusters and connector health using monitoring tools and OpenShift native tools.
• Build and maintain custom connectors for specialized data sources and sinks.
• Support production deployments and troubleshoot across Kafka pipelines and OpenShift.
• Understand cloud managed services (AWS, Azure) for Confluent connectors and data integrations.
• Coordinate with offshore teams and conduct code reviews and quality checks.

Skills

Kafka

OpenShift

Docker

Kustomize

CI/CD

GitHub Actions

Argo CD

Monitoring

Python

Java

Education

Bachelor's degree in Computer Science, Engineering, or related technical field

Tools

Confluent Kafka

OpenShift

Docker

Kustomize

GitHub Actions

Argo CD

Prometheus

Grafana

Dynatrace

Splunk

Confluent Control Center

Job description

AEP is seeking a Senior Kafka Engineer to design, deploy, and operate the enterprise Kafka platform that forms the data movement backbone. The role focuses on stability, scalability, data safety, and enabling producer/consumer teams.

You will containerize services on OpenShift, implement CI/CD with GitHub Actions and Argo CD, perform Kafka administration, and collaborate with architects and business teams to deliver scalable, cloud-integrated data pipelines.