Kafka Infra Engineer - Remote & Growth

AgileEngine, LLC.; Madrid; Hybrid; EUR 60.000 - 90.000; Vollzeit

Benefits dieser Stelle

Professional growth

Competitive compensation

Exciting projects

Flextime

Zusammenfassung

AgileEngine is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind our event-driven integration platform, including build, deployment, and monitoring infrastructure. You will design topic naming, partitioning, retention, and broker configurations to support reliable, scalable data flows.

You will implement at-least-once and exactly-once semantics, build streaming observability, plan capacity, and steward CI/CD pipelines (Jenkins) with containerized deployments.

Qualifikationen

• 4+ years of production Kafka experience required.
• Strong Linux administration and automation skills in production.
• Proficient in Python scripting for automation and tooling.
• Experience owning CI/CD pipelines and release management.
• Hands-on containerization and orchestration with Docker/Podman and Kubernetes.
• Experience with IaC and configuration management tools.
• Solid observability, monitoring and alerting practices.
• Experience with AI coding tools and adoption in daily work.
• Knowledge of networking, TLS and enterprise authentication.
• Production operations mindset: incident response and capacity planning.

Aufgaben

• Own the Kafka estate: topic design, partitioning, retention, and broker config.
• Ensure platform resilience during vendor outages and traffic spikes.
• Implement at-least-once and exactly-once semantics and dead-letter handling.
• Build streaming observability: lag, throughput, dashboards, and SLOs.
• Plan capacity and retention for durable logs with regulatory needs.
• Evaluate vendor streaming and CDC mechanisms as the platform matures.
• Own build/release pipeline: Jenkins pipelines, signed container images, docs.
• Operate runtime estate: containerized deployments and gateway hosts.
• Automate provisioning and keep configuration as truth source.
• Partner with Python developers on deployability, performance and cost.

Kenntnisse

Kafka

Linux engineering

Python scripting

CI/CD pipelines

Containerization

Infrastructure as code

Observability

AI coding tools

Networking & TLS

Incident response

Communication

Tools

Jenkins

Docker

Podman

Kubernetes

OCI images

Terraform

Jobbeschreibung

AgileEngine is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind our event-driven integration platform, including build, deployment, and monitoring infrastructure. You will design topic naming, partitioning, retention, and broker configurations to support reliable, scalable data flows.

You will implement at-least-once and exactly-once semantics, build streaming observability, plan capacity, and steward CI/CD pipelines (Jenkins) with containerized deployments.