Senior Kafka Platform Engineer — Real-Time Data Backbone

AEP; Columbus (OH); Híbrido; USD 99.000 - 129.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Generous time off

Educational opportunities

Descripción de la vacante

AEP is seeking an experienced Kafka Platform Engineer to own and operate the enterprise Kafka backbone, focusing on stability, scalability, and data safety. The role involves deploying connectors, containerizing services on OpenShift, and building automated deployment pipelines.

You will work with architects and product teams to deliver scalable data integrations, monitor clusters with industry tools, and support production deployments across cloud and on-prem environments.

Formación

• Strong hands-on Kafka experience in production environments.
• Experience with Kubernetes/OpenShift deployments and multi-region HA.
• Experience building producer/consumer applications using Kafka connectors.
• Experience with Python/Java development.

Responsabilidades

• Deploy and manage Kafka connectors (CDC, JDBC, IBM MQ, Redshift, etc.) in Confluent Kafka environments.
• Containerize and deploy microservices and connectors on OpenShift using Docker, YAML, and Kustomize.
• Implement CI/CD pipelines using GitHub Actions and Argo CD for automated deployments.
• Perform basic Kafka administration tasks such as topic creation, partitioning, replication, and monitoring.
• Collaborate with architects and business teams to understand integration requirements and deliver scalable solutions.
• Monitor Kafka clusters and connector health using tools like Confluent Control Center, Prometheus, Grafana, Dynatrace, Splunk and OpenShift-native monitoring.
• Build and maintain custom connectors for specialized data sources and sinks.
• Support production deployments and troubleshoot issues across Kafka pipelines and OpenShift environments.
• Clear understanding of cloud managed services including AWS, Azure cloud to deploy Confluent connectors for data integrations.
• Support real-time data streaming between on-prem systems and cloud/SaaS platforms.
• Work closely with business teams, architects, and product owners to understand integration requirements.
• Coordinate with co-sourced/offshore teams for development and deployment tasks.
• Conduct code reviews, quality checks, and ensure adherence to standards.
• Ensure reliability and scalability of Kafka pipelines by applying 12-Factor principles and microservices architecture.
• Collaborate with architects and platform teams to align Kafka solutions with enterprise integration patterns and cloud strategies.
• Automate deployment, monitoring, and operational tasks wherever possible.
• Own integration of deliverables from design through production deployment.
• Act as Subject Matter Expert (SME) for Kafka-based integrations and cluster administration.

Conocimientos

Kafka

Distributed systems

OpenShift

CI/CD pipelines

Python/Java

Cloud deployments

Containerization

Monitoring & observability

Educación

Bachelor's degree in CS/engineering or related field

Herramientas

Confluent Kafka

OpenShift

Docker

Kustomize

GitHub Actions

Argo CD

Prometheus

Grafana

Dynatrace

Splunk

Confluent Control Center

Descripción del empleo

AEP is seeking an experienced Kafka Platform Engineer to own and operate the enterprise Kafka backbone, focusing on stability, scalability, and data safety. The role involves deploying connectors, containerizing services on OpenShift, and building automated deployment pipelines.

You will work with architects and product teams to deliver scalable data integrations, monitor clusters with industry tools, and support production deployments across cloud and on-prem environments.