Kafka Infrastructure Engineer – Remote/Hybrid

AgileEngine, LLC.; Málaga; Hybrid; EUR 107,000 - 151,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind our event-driven integration platform, including build, deployment and monitoring infrastructure. You will work with a global team to ensure resilience and scalability.

You will design topic and partition strategies, manage replication and retention, implement at-least-once and exactly-once semantics, and contribute to a modern CI/CD pipeline using Jenkins, Docker/Podman, and IaC tooling.

Qualifications

• 4+ years of production Kafka experience with topics, partitions and consumer groups.
• Strong Linux engineering and automation using Python.
• Ownership of CI/CD pipelines and release management.
• Experience with containerization and orchestration (Docker/Podman, Kubernetes).
• Experience with infrastructure-as-code and configuration management.
• Observability tooling, metrics, logs and SLOs.
• Familiarity with AI-assisted development tools is a plus.

Responsibilities

• Own the Kafka estate: topic design, partitioning, replication and retention.
• Ensure platform resilience and handle load spikes and outages.
• Implement at-least-once and exactly-once semantics, dead-letter handling.
• Build streaming observability: lag, throughput, tracing, dashboards, alerts.
• Plan capacity, cost and retention for durable logs with regulatory retention.
• Integrate vendor-native streaming and CDC mechanisms as needed.
• Own and evolve the build/release pipeline with Jenkins and container images.
• Operate containerised deployments and modernise the OS of the estate.
• Automate provisioning and maintain single-source configuration inventory.
• Collaborate with Python developers on deployability, performance and cost.

Skills

Kafka production experience

Python scripting

CI/CD pipelines ownership

Docker/Podman

Kubernetes

Infrastructure-as-code

Observability

AI-assisted development

OAuth2 / security

Production operations

Linux engineering

Communication

Tools

Jenkins

Docker

Podman

Kubernetes

Terraform

Job description

AgileEngine is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind our event-driven integration platform, including build, deployment and monitoring infrastructure. You will work with a global team to ensure resilience and scalability.

You will design topic and partition strategies, manage replication and retention, implement at-least-once and exactly-once semantics, and contribute to a modern CI/CD pipeline using Jenkins, Docker/Podman, and IaC tooling.