Senior DevOps Engineer (all genders) | Data Platform

Meyandy LLC; Deutschland; Hybrid; EUR 110.000 - 150.000; Vollzeit

Zusammenfassung

Real Digital is seeking a Senior DevOps Engineer (Data Platform) to own our self-hosted Kafka backbone, Debezium-based CDC pipelines, and Kubernetes workloads. You will manage ACLs, certificates, upgrades, and disaster recovery while driving automation across GitLab CI/CD and IaC tooling.

You’ll collaborate with senior data engineers on design reviews and longer-term BigQuery architecture initiatives. You should have strong Python and SQL skills, plus experience with cost-aware data operations,

Qualifikationen

• Several years of experience operating production infrastructure, with on-call rotation.
• Deep hands-on Kafka broker operations experience including sizing, rebalancing, replication, and upgrades.
• Experience with cross-system data replication (Debezium / Kafka Connect) in production.
• Strong production experience with Kubernetes, resource management and autoscaling.

Aufgaben

• Own and operate the self-hosted Kafka backbone and Schema Registry end-to-end.
• Manage Debezium-based CDC pipelines and BigQuery sinks.
• Run workloads on Kubernetes and manage Navarch workflow tasks.
• Automate access management and IaC across GitLab CI/CD pipelines.
• Maintain observability with Datadog, alerting, and cost controls.

Kenntnisse

Kafka operations

Kubernetes

Python

Terraform

Ansible

BigQuery

SQL

Tools

GitLab CI/CD

Terraform

Ansible

Pulumi

Helm

Kubernetes

BigQuery

Jobbeschreibung

Anstellungsdetails Permanent Contract, Full- or Part-Time, Remote or Cologne / Düsseldorf / Darmstadt / Berlin

Info

Who we are We are the tech powerhouse behind Kaufland’s international online marketplaces and a company within the Schwarz Group. Driven by a start-up mentality and working on an equal footing within flat hierarchies, we develop the best marketplace solutions for our customers, sellers and partners. #FromEuropeForEurope Day by day, our Tech & Product Team of about 400 experts pursues the goal of creating the best possible customer shopping experience for our online marketplace. Through innovative technical developments, they not only create an outstanding shopping experience but also lay the foundation for an optimal selling experience for our sellers. Learn more about our division, areas, and cross-functional teams here!

Aufgaben

Your tasks – this is what awaits you in detail

• You run and own our self-hosted Apache Kafka backbone and Schema Registry end-to-end, managing ACLs, client quotas, certificates, upgrades, and disaster recovery
• You operate our Debezium-based change data capture pipelines, overseeing connectors, snapshots, offsets, replica lag, and BigQuery sink loaders
• You run our workloads on Kubernetes, including Navarch - our self-built workflow orchestrator executing ~7,500 SQL and Python tasks daily - owning resource management, autoscaling, and debugging
• You automate and own access management and infrastructure as code across our self-hosted GitLab CI/CD pipelines, replacing manual tickets with code-driven policies
• You own observability and infra costs - Datadog across Kafka and sink loaders, Slack-native alerting, and maintaining an honest view of K8s, Kafka retention, and log volume expenses
• You carry your share of triage, one week in three - handling ingestion and access requests, schema alerts, and supporting analysts, while helping automate this layer via AI-assisted review
• You act as a senior sparring partner alongside two senior data engineers (reporting to the Head of Data & Analytics) - driving design reviews and multi-quarter initiatives, with room to grow into BigQuery architecture over time: access frameworks, partitioning and slot strategy, the pipelines and models on top
• Optional: You take part in our 24/7 on-call rotation

Profil

Senior DevOps Engineer (all genders) | Data Platform — Real Digital.

Profil Your profile – this is what we expect from you

• You have several years of experience operating production infrastructure you were personally responsible for, including on-call rotation. You have run systems that broke, can explain what went wrong and how you permanently fixed it, and can detail a specific trade-off you navigated backed by concrete numbers
• You have deep hands-on expertise in Apache Kafka broker operations, including cluster sizing, partition rebalancing, replication, ISR behavior, and version upgrades
• You have operated cross-system data replication in production - ideally Debezium on Kafka Connect, or database replication via binlog, GTID, WAL, or agent-based pipelines
• You demonstrate strong production experience with Kubernetes, master resource management and autoscaling, and possess sharp debugging reflexes for complex failure modes
• You manage infrastructure and access as code across multiple environments (ideally Terraform and Ansible; Pulumi, Helm or serious configuration management also counts) following a strict least-privilege IAM approach across multiple environments, ideally with GCP experience or transferable AWS/Azure knowledge
• You use production-grade Python and SQL as practical tools, applying a cost-conscious mindset to query performance, scanning volume, and data pruning
• You thrive in a low-process, Kanban-driven environment where seniors self-organize, rotate weekly triage duties without micro-management, and prefer automating recurring requests over manually repeating them
• Optional: You bring skills or interest in object-oriented design, data modeling/dbt, BigQuery architecture, orchestration frameworks (Airflow, Dagster), stream processing …

Mots-clés : Data Engineering.