Data Engineer (Analytics & Products)

kleinanzeigen.de GmbH; Germania; Ibrido; EUR 90.000 - 120.000; Tempo pieno

Vantaggi offerti da questo lavoro

Private health insurance

Gym Pass

Remote work allowance

Personal development budget

Flexible holidays

Descrizione del lavoro

Kleinanzeigen sucht eine erfahrene Data-Engineering-Fachkraft zur Weiterentwicklung unserer data-Stacks. Sie arbeiten an dbt-Modellen, Airflow-Pipelines und modernen Lakehouse-Architekturen in einem hybriden Arbeitsmodell in Deutschland.

Fokus liegt auf cost-aware Modeling, Governance und der Bereitstellung zuverlässiger Daten für Analytik-Teams. Sie arbeiten eng mit Analysts und Product-Teams zusammen, gestalten Datenmodelle eigenständig, definieren Metriken und Qualitätschecks und tragen zu

Competenze

• Fundierte SQL-Kenntnisse mit Fokus auf Performance und Tuning.
• Nachweisbare Erfahrung mit dbt in einer Medallion-Architektur.
• Erfahrung mit Airflow-DAGs und Python-Operatoren.
• Cloud-/Data-Warehouse-Kenntnisse (Delta Lake, Parquet, Avro).
• Verständnis von Data Governance und cost-aware Modelling.

Mansioni

• Implementierung und Weiterentwicklung von dbt-Modellen in der Medallion-Architektur.
• Aufbau und Refactoring von Modellen für Marketing, Produktmetriken und Transaktionen.
• Schreiben rekurrenter Makros, YAML-Dokumentation und Metadaten-Tagging.
• Entwurf homogener Datenmodelle für effiziente Abfragen und Kostensteuerung.
• Aufbau robuster Data-Pipelines mit Airflow und DS-/SLO-Definitionen.

Conoscenze

Advanced SQL

dbt

Airflow

Python

Delta Lake

Parquet

Avro

Looker LookML

Kubernetes

Cost optimization

Strumenti

dbt

Airflow

Python

Delta Lake

Parquet

Avro

Descrizione del lavoro

What will you do

• Implement dbt models across the medallion architecture applying the right materialisation strategy for each layer and use case — incremental, full refresh or snapshots — with consistent naming conventions, YAML documentation, metadata tagging, unit tests to validate critical business logic, and reusable macros for common transformation and data replication patterns
• Build and refactor models for different business areas of the company, including for example marketing performance, product metrics, C2C transactions, monetisation, vibrancy, and trust & safety
• Author reusable macros and apply consistent naming conventions, YAML documentation, and metadata tagging (billed, retention, gdpr)
• Design data models that are homogeneous with the existing stack, built at the right grain for the use case, cost-effective to run, and self-discoverable without needing the author to explain them
• Perform cost-aware modelling: clustering strategies, warehouse sizing, incremental scan reduction, and pre-aggregation layers
• Build and maintain Airflow DAGs using Python operators, designing DAG topology deliberately by choosing execution patterns, dependency structures, and sensor logic that match the operational requirements of each pipeline
• Design sensor logic for pipeline dependencies, including intraday vs daily completeness checks and DST-aware temporal handling
• Operate Cosmos dbt task groups within Airflow, including DAG splitting, warehouse selection, and Cosmos version upgrades
• Integrate new data sources end-to-end by building reliable, fault-tolerant connections across heterogeneous endpoint types including REST APIs, event streams, database connectors and file-based sources, with error handling, retry logic, and security patterns that make each integration production-safe from day one, unit testing every operator and transformation component where possible, and defining SLAs and SLOs that set clear expectations on data freshness, completeness, and availability for downstream consumers
• Work with data in the right format for each layer: Avro for event-driven ingestion schemas, Parquet for efficient columnar storage, and Delta for ACID-compliant lakehouse tables with time travel and schema evolution

Data Modelling & dbt Development

• Implement dbt models across the medallion architecture applying the right materialisation strategy for each layer and use case — incremental, full refresh or snapshots — with consistent naming conventions, YAML documentation, metadata tagging, unit tests to validate critical business logic, and reusable macros for common transformation and data replication patterns
• Build and refactor models for different business areas of the company, including for example marketing performance, product metrics, C2C transactions, monetisation, vibrancy, and trust & safety
• Author reusable macros and apply consistent naming conventions, YAML documentation, and metadata tagging (billed, retention, gdpr)
• Design data models that are homogeneous with the existing stack, built at the right grain for the use case, cost-effective to run, and self-discoverable without needing the author to explain them
• Perform cost-aware modelling: clustering strategies, warehouse sizing, incremental scan reduction, and pre-aggregation layers

Data Ingestion & Pipeline Engineering

• Build and maintain Airflow DAGs using Python operators, designing DAG topology deliberately by choosing execution patterns, dependency structures, and sensor logic that match the operational requirements of each pipeline
• Design sensor logic for pipeline dependencies, including intraday vs daily completeness checks and DST-aware temporal handling
• Operate Cosmos dbt task groups within Airflow, including DAG splitting, warehouse selection, and Cosmos version upgrades
• Integrate new data sources end-to-end by building reliable, fault-tolerant connections across heterogeneous endpoint types including REST APIs, event streams, database connectors and file-based sources, with error handling, retry logic, and security patterns that make each integration production-safe from day one, unit testing every operator and transformation component where possible, and defining SLAs and SLOs that set clear expectations on data freshness, completeness, and availability for downstream consumers
• Work with data in the right format for each layer: Avro for event-driven ingestion schemas, Parquet for efficient columnar storage, and Delta for ACID-compliant lakehouse tables with time travel and schema evolution

Data Quality & Reliability

• Write dbt tests (not_null, unique, accepted_values) and semantic row-count checks to catch data quality issues before they reach consumers
• Author unit tests for critical business logic using dbt's unit test framework
• Own data quality across the full lifecycle: design and maintain quality check pipelines combining dbt tests, semantic validation and custom logic, monitor pipeline health proactively before issues reach downstream consumers, diagnose and resolve root causes across schema drift, silent failures, duplicate data and idempotency issues, and plan and execute backfills safely with pre-validation, before and after checks and documented rollback criteria
• Apply governance standards consistently across every pipeline and data asset you own, ensuring compliance requirements, data classification, retention policies and ownership accountability are traceable and maintained as the platform evolves
• Contribute to the reliability of the platform beyond your own work, flagging systemic issues, improving shared patterns, and leaving every codebase and dataset in better shape than you found it

Cost Optimisation

• Identify and implement cost reduction opportunities: warehouse downsizing, incremental model rewrites, DAG consolidation, unused table deprecation
• Ensure governance rules for compliance and cost ownership are applied consistently across pipelines and data assets — including tagging standards (business_unit, business_area, team) in Airflow and dbt
• Evaluate and act on compute cost signals from Databricks query cost tables

Stakeholder Collaboration & Knowledge Sharing

• Partner with analysts, product managers and domain teams to translate business requirements into accurate, reusable data models, and enable them to build and own their own data assets through patterns, templates, PR reviews and pair programming
• Provide consultancy and training to product teams on data ingestion and transformation patterns
• Support analysts in adopting dbt for report-layer development, including onboarding, PR review, and pair programming
• Enable teams to build their own data products: provide patterns, templates, and consultancy so analysts and product teams can develop and own report-layer assets independently
• Define and document reusable metrics, dimensions and business logic in the semantic layer, and capture institutional knowledge including domain assumptions, architectural decisions and data lineage context so it lives in the codebase and not in people's heads
• Participate in cross-team alignment on data model standards, shared definitions and platform evolution, representing the data engineering perspective in decisions that affect multiple domains

AI-First Engineering

• Review and verify all AI-generated code before submitting for peer review — you own correctness, not the AI
• Maintain machine-readable context files (CLAUDE.md) in repositories you own, capturing architectural constraints, naming conventions, and domain-specific rules for AI agents
• Use AI tooling as a force multiplier: author intent specifications, verify AI-generated outputs with full ownership of correctness, contribute to shared skills and context files, and use automation to raise quality standards across the team
• Build and iterate on Claude agents for scoped engineering tasks (e.g. report-layer model extension, documentation generation, comparison notebooks)
• Orchestrate multi-agent workflows for complex tasks (large-scale migrations, documentation backfills, cross-domain refactors) while verifying outcomes
• Contribute to the team's AI Skill Library: dbt patterns, ingestion templates, data quality check agents

What we´re looking for

Required

• Advanced SQL: window functions, CTEs, query optimisation, partitioning and clustering, with enough depth to understand execution plans and diagnose performance issues
• Hands‑on production experience with dbt across the medallion architecture: models, tests, unit tests, macros, snapshots, YAML documentation and materialisation strategies
• Experience building and operating Airflow DAGs using Python operators, with the ability to design DAG topology deliberately for different pipeline requirements
• Proficiency in Python and shell scripting for pipeline development, ingestion tasks and operational tooling
• Solid understanding of lakehouse concepts and data formats: Delta Lake for ACID‑compliant storage, Parquet for columnar efficiency, and Avro for event‑driven schema definition
• Ability to work independently: owning tickets from design through delivery, including incident response and backfills
• Ability to gather requirements from non‑technical stakeholders, understand consumption patterns, and translate business needs into data models that are accurate, maintainable and cost‑effective
• Experience defining and maintaining schema definitions (Avro/ODIN or equivalent schema registry)
• Fluency with AI‑assisted development workflows as part of daily practice: authoring intent specifications, verifying and owning AI‑generated outputs, and contributing to shared context and tooling so the whole team benefits
• Experience with semantic layer concepts: defining reusable metrics, dimensions, and business logic that serve as a single source of truth across consumers (e.g. dbt Semantic Layer, Looker LookML, or equivalent)
• Spark comprehension at the level needed to understand how Databricks processes dbt models, read execution plans, and diagnose and fix performance issues such as full table scans, merge inefficiencies and file fragmentation
• Engineering fundamentals applied to data work: modularity, idempotency, testability and single responsibility as default practices, not afterthoughts
• Experience treating cost as a first‑class constraint: identifying and acting on optimisation opportunities across compute, storage and pipeline design

Nice to Have

• Experience with Kafka or event‑driven ingestion
• Familiarity with Salesforce, Google Ads, Meta, or other marketing/CRM APIs
• Experience with data modelling patterns and when to apply each: dimensional modelling, star schema, wide tables, one big table, slowly changing dimensions, and normalisation vs denormalisation tradeoffs
• Exposure to compliance requirements in data engineering: data classification, retention policies and privacy‑aware modelling
• Experience with cost optimisation in cloud data platforms
• Experience orchestrating multi‑agent AI workflows for complex engineering tasks
• Contribution to shared AI tooling, skills, or context files in previous roles

What Makes You a Great Fit

• Proactive by default: you don't wait to be asked. You flag issues before they become incidents, own follow‑ups after delivery, and look for ways to improve things beyond the scope of your ticket.
• Generous with knowledge: you enjoy sharing what you know, whether that's through documentation, pair programming, code reviews, or supporting analysts and product teams in adopting the data platform.
• A team builder: you care about the people around you as much as the work. You contribute to a culture where the team gets better together, not just individually.
• An architect of intent, not just a code translator: you define the what and why with precision, use AI to handle the how, and verify that the output actually solves the problem. You understand that AI commoditises syntax production — your value is in T‑shaped thinking that connects business context, data modelling, and technical constraints.
• Quality‑first mindset enabled by AI: you treat comprehensive testing, documentation, and data quality checks as non‑negotiable — precisely because AI has made them cheap to produce. Low test coverage is no longer a time constraint; it’s a choice.

At Kleinanzeigen, we care about our people — their wellbeing, growth, and life beyond work. Our benefits are designed to support you holistically, giving you flexibility and opportunities to thrive.

Flexibility

• 26 business days of paid annual holidays
• 20 days “work from anywhere”
• Flexible working time (hybrid)
• Paid additional parental leave for both parents
• Extra paid days off for special occasions (e.g. your wedding or a move)

Wellbeing & Support

• Private health insurance
• Gym Pass (may cover up to 70% of gym membership fees)
• 40€ per month as a remote working allowance.

Growth & Development

• Personal development budget
• Learning opportunities to grow your skills and career
• A unique position in an international and dynamic environment
• A flat, collaborative culture where your feedback actively shapes our processes

Perks & Everyday Benefits

• Competitive compensation package (Base salary + annual bonus)
• Flexible Remuneration Program (tax‑efficient options for transport, restaurant vouchers and childcare)
• Corporate benefits and exclusive discounts
• An attractive referral program

Kleinanzeigen is an equal opportunity employer that values diversity and inclusion. We welcome people of all backgrounds, races, religions, colours, national origins, gender identities, sexual orientation, ages, marital status, or disability status. If this role sparks your interest but you’re not sure you meet every single requirement, we’d still love to hear from you anyway!