Kafka Infrastructure Engineer — Remote & Flexible

AgileEngine, LLC.; Perímetro Urbano Barranquilla; Hybrid; COP 381,934,000 - 572,902,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine, LLC. seeks an Infrastructure Engineer (Kafka) to own and operate the Kafka estate for a high‑scale event‑driven integration platform. You will build, deploy, and monitor the infrastructure, ensuring reliability and scalability across the enterprise.

You will design topics/partitions, implement exact-once semantics, and lead CI/CD with Jenkins, collaborating with Python developers and global teams. Strong Linux and observability skills are essential.

Qualifications

• Production Kafka experience of 4+ years and strong design of topics, partitions, and replication.
• Experience with Linux engineering and Python scripting.
• Hands-on CI/CD ownership, preferably Jenkins declarative pipelines.
• Experience with containers and orchestration (Docker/Podman, Kubernetes).
• Infrastructure-as-code and configuration management.
• Observability: log/metrics, alerting, SLOs.
• GenAI tooling fluency in coding/workflows.
• Networking, TLS, and authentication (OAuth2).
• Incident response, capacity planning, and change control.

Responsibilities

• Own the Kafka estate: topic naming, partitioning, replication, retention, and broker config upgrades.
• Ensure platform resilience during upstream SaaS outages and handle traffic spikes while keeping events replayable and auditable.
• Implement at-least-once and exactly-once semantics, idempotent producers, and dead-letter handling with replay tooling.
• Build streaming observability: lag, queue depth, throughput, end-to-end tracing, dashboards, and alerts.
• Plan capacity and retention for durable log with regulatory audit requirements and optimize as adoption grows.
• Evaluate vendor-native streaming and CDC mechanisms for ingress/egress paths.
• Own and evolve the build/release pipeline: Jenkins pipelines, packaged installs for Linux, signed images, and artifact distribution.
• Operate runtime estate: container deployments, gateway hosts, OS modernization.
• Automate environment provisioning and maintain a single source of truth for config/assets.
• Collaborate with Python developers on deployability, performance, and cost; support production incidents.

Skills

Kafka

Linux

Python

CI/CD

Docker

Kubernetes

IaC

Observability

AI/GenAI

Tools

Docker

Podman

OCI images

Jenkins

Job description

AgileEngine, LLC. seeks an Infrastructure Engineer (Kafka) to own and operate the Kafka estate for a high‑scale event‑driven integration platform. You will build, deploy, and monitor the infrastructure, ensuring reliability and scalability across the enterprise.

You will design topics/partitions, implement exact-once semantics, and lead CI/CD with Jenkins, collaborating with Python developers and global teams. Strong Linux and observability skills are essential.