Kafka Engineer

Your ITeams Sp. z o.o.; Deutschland; Hybrid; EUR 90.000 - 130.000; Vollzeit

Benefits dieser Stelle

Luxmed Gold Extended medical care

Multisport Plus

Zusammenfassung

Your ITeams Sp. z o.o. seeks an experienced Kafka Engineer to design, build and operate a hybrid Apache Kafka environment spanning on-premises and AWS.

You will work with Confluent Platform (Enterprise) and Confluent Cloud to implement reliable replication, automated pipelines and observability across distributed systems. Responsibilities include deploying Kafka-based solutions on AWS, performing migrations, and creating CI/CD pipelines with IaC.

Qualifikationen

• Minimum 4 years hands-on experience with Apache Kafka.
• Proven experience with Confluent Platform (Enterprise) and Confluent Cloud.
• Experience with cluster migration and replication tools (MirrorMaker 2, Cluster Linking, Confluent Replicator).
• Practical AWS experience (VPC, IAM, EC2, EKS, MSK, S3, KMS).
• Experience designing hybrid on-prem ↔ AWS environments.
• Terraform experience including module development and testing (e.g., Terratest, tfsec).
• Familiarity with CI/CD platforms (GitHub Actions, GitLab CI, Jenkins).
• Proficiency in Python, Ansible and Bash scripting.
• Hands-on observability tooling (Prometheus, Grafana, OpenTelemetry, Elastic Stack).
• Security concepts for messaging platforms (TLS, SASL, ACLs, RBAC).
• Agile methodologies (Scrum/Kanban); Jira and Confluence knowledge.

Aufgaben

• Design, deploy and operate Kafka-based solutions on AWS and on-premises.
• Implement and maintain data replication across clusters using MirrorMaker 2 and Confluent Replicator.
• Create CI/CD pipelines and IaC to provision cloud Kafka infrastructure.
• Lead cluster migrations ensuring data integrity and minimal disruption.
• Develop automation for Kafka integration tests and deployment validation.
• Implement monitoring and observability for Kafka and related services.
• Apply security best practices for Kafka and cloud services.
• Collaborate with cross-functional teams in Agile environments.

Kenntnisse

Apache Kafka

Confluent Platform

Confluent Cloud

MirrorMaker 2

Cluster Linking

Confluent Replicator

Hybrid cloud

Python

Security concepts for messaging

Agile/Scrum/Kanban

Jira/Confluence

Prometheus

Grafana

OpenTelemetry

AWS (VPC/IAM/EC2/EKS/MSK/S3/KMS)

Terraform

CI/CD (GitHub Actions/GitLab/Jenkins)

Tools

Terraform

GitHub Actions

GitLab CI

Jenkins

Kubernetes

MSK

EKS

Jobbeschreibung

We are seeking an experienced Kafka Engineer to design, build and operate a hybrid Apache Kafka environment spanning on-premises and AWS. The person in this role will work with Confluent Platform (Enterprise) and Confluent Cloud to implement reliable, secure replication, automated pipelines and observability across distributed systems. This role is ideal for someone who enjoys hands-on engineering, automation, and collaboration across infrastructure and development teams.

Responsibilities

• Design, deploy and support Kafka-based solutions on AWS (including MSK, EKS, Aurora and related services).
• Implement and maintain data replication between on-premises and AWS Kafka clusters using MirrorMaker 2, Cluster Linking and Confluent Replicator.
• Create and operate CI/CD pipelines and Infrastructure as Code (IaC) practices to provision and manage cloud Kafka infrastructure.
• Lead cluster migrations and replication projects, ensuring data integrity, performance and minimal disruption.
• Develop automation for Kafka integration tests and continuous validation of deployments.
• Implement and maintain monitoring and observability for Kafka and related services using tools such as Prometheus, Grafana, Confluent Control Center, OpenTelemetry and the Elastic Stack.
• Apply security best practices for Kafka and cloud services: TLS, SASL, ACLs, RBAC and encryption at rest and in transit.
• Collaborate with cross-functional teams in Agile settings, contributing to design reviews, runbooks and incident response.

Qualifications

• Minimum 4 years of hands-on experience with Apache Kafka.
• Strong expertise with Confluent Platform (Enterprise) and Confluent Cloud.
• Proven experience with cluster migration and replication technologies such as MirrorMaker 2, Cluster Linking and Confluent Replicator.
• Practical experience with AWS services including VPC, IAM, EC2, EKS, MSK, S3, KMS and Secrets Manager.
• Experience designing, implementing and maintaining hybrid environments (on-premises ↔ AWS).
• Terraform experience preferred, including module development and testing (e.g., Terratest, tfsec).
• Familiarity with CI/CD platforms such as GitHub Actions, GitLab CI or Jenkins.
• Proficiency in scripting and automation with Python, Ansible and Bash.
• Hands-on experience with monitoring and observability tooling as listed above.
• Strong understanding of security concepts relevant to messaging platforms.
• Experience working in Agile teams using Scrum and/or Kanban, and practical knowledge of Jira and Confluence.
• Unique TEAL culture, relationship- and respect-driven community, non-corporate atmosphere.
• Agile approach and no bureaucracy.
• Outstanding integration trips to various places in Europe.
• Activities to support your well-being and health.
• Luxmed Gold Extended medical care and Multisport Plus benefit.