Kafka Infrastructure Engineer — Remote & Flexible Hours

AgileEngine, LLC.; Granada; Hybrid; EUR 134,000 - 170,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine, LLC. is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind an event-driven integration platform, including build, deployment, and monitoring infrastructure.

You will design and manage topic and partition strategies, implement exactly-once/at-least-once semantics where required, and drive reliability through observability, automation, and IaC. Collaboration with global teams and CI/CD ownership are essential.

Qualifications

• Deep, hands-on production Kafka experience of at least 4+ years.
• Practical understanding of event-driven delivery semantics: at-least-once versus exactly-once.
• Strong enterprise Linux engineering and automation - shell plus Python scripting.
• CI/CD pipeline ownership, ideally Jenkins declarative pipelines, with real artifact and release management experience.
• Containers - Docker or Podman, OCI images - and container orchestration platforms.
• Infrastructure-as-code and configuration management.
• Observability tooling: log aggregation, metrics, alerting and SLO definition.
• GenAI-assisted development and daily workflow adoption.

Responsibilities

• Own the Kafka estate - topic design and naming standards, partitioning and keying strategy, replication, retention and compaction policy, consumer group and offset management, quotas, and broker configuration and upgrades.
• Ensure platform resilience and reliability - keep the integration platform available when upstream vendor SaaS is degraded, absorb traffic spikes, and ensure events remain replayable and auditable.
• Engineer delivery guarantees: at-least-once and exactly-once semantics, idempotent producers, transactional writes where warranted, dead-letter topics, and self-service replay tooling.
• Build streaming observability: consumer lag, queue depth, throughput, and end-to-end tracing; dashboards, SLOs and alerts.
• Plan capacity, cost and retention for durable logs with regulatory audit requirements.
• Evaluate and integrate vendor-native streaming and change-data-capture mechanisms.
• Own and evolve build and release pipeline - automated Jenkins pipelines producing packaged installs for multiple Enterprise Linux generations, signed container images, and generated docs.
• Operate the runtime estate: containerised deployments, gateways, and OS modernization.
• Automate environment provisioning and configuration, maintain single source of truth for assets.
• Partner with Python developers on deployability, performance and cost; contribute to production support.

Skills

Kafka

Python scripting

CI/CD pipelines

Docker/Podman

Infrastructure-as-code

Observability

GenAI adoption

TLS/OAuth2

Incident response

Tools

Jenkins

Kubernetes

Job description

AgileEngine, LLC. is seeking an Infrastructure Engineer (Kafka) to own and operate the Kafka estate behind an event-driven integration platform, including build, deployment, and monitoring infrastructure.

You will design and manage topic and partition strategies, implement exactly-once/at-least-once semantics where required, and drive reliability through observability, automation, and IaC. Collaboration with global teams and CI/CD ownership are essential.