Senior Data Engineer

Jobgether

Deutschland

Remote

EUR 91,000 - 127,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Fully remote in Europe

Job summary

Jobgether is seeking a Senior Data Engineer based in Germany to own a data platform that ingests from multiple sources and delivers reliable company intelligence. You will architect, implement, and operate pipelines spanning ingestion, processing, storage, and production.

You will design scalable, event-driven as well as batch workflows, ensuring data quality and monitoring across environments. You will work with modern tech including LLM-powered enrichment, dbt, and cloud-based storage, while

Qualifications

  • 5+ years of experience building and operating production data platforms with multi-source ingestion and ownership.
  • Production experience with workflow orchestration tools (Airflow preferred; Dagster/Prefect/Temporal also relevant).
  • Hands-on experience with dbt and Google Cloud Platform (GCP).
  • Strong Python programming skills, including async HTTP requests and concurrent processing.
  • Advanced SQL and PostgreSQL skills, including data modeling and incremental loading.
  • Experience designing and operating event-driven processing systems and data pipelines.
  • Knowledge of data contracts, raw/curated storage, retention, automated testing, and monitoring.
  • Ability to design resilient ingestion and transformation processes for consistency and recovery.
  • Experience building data platforms or large pipelines from scratch is a strong advantage.
  • Familiarity with web scraping, entity resolution, data enrichment with LLMs is beneficial.
  • Experience in small teams or startups is advantageous.

Responsibilities

  • Design and develop a multi-source data platform with concurrent ingestion and orchestration.
  • Build reliable ingestion systems to provider APIs, webhooks, files, and web scraping with robust error handling.
  • Implement scheduled workflows and backfills using Airflow/Dagster/Prefect.
  • Develop event-driven processing using messaging queues with retries and DLQ.
  • Store original payloads in cloud storage and use dbt to transform into structured layers.
  • Ensure idempotent upserts, data contracts, validation, testing, and monitoring.
  • Design and optimize PostgreSQL schemas and loading strategies for performance and scale.
  • Integrate LLM-based enrichment and structured data extraction capabilities.
  • Take end-to-end ownership from architecture to deployment and production reliability.

Skills

Python
Asynchronous HTTP requests
Concurrent processing
SQL
PostgreSQL
Data modeling
Incremental loading
Data contracts
Automated testing
Pipeline monitoring
LLM enrichment

Tools

Airflow
Dagster
Prefect
Temporal
dbt
Google Cloud Platform

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer based in Germany.

This role offers the opportunity to take ownership of a data platform that transforms information from multiple sources into reliable, continuously updated company intelligence. You will lead the platform's development from architecture and ingestion through processing, storage, and production operations. Working with diverse data sources, you will design scalable pipelines that combine event-driven processing with orchestrated batch workflows. You will also establish robust data quality, validation, and monitoring practices to ensure information remains accurate, consistent, and accessible. The position combines hands-on engineering with architectural responsibility, offering the opportunity to build substantial data infrastructure and integrate modern technologies, including LLM-powered enrichment and extraction.

Accountabilities
  • Design and develop a multi-source data platform supporting concurrent ingestion, event-driven processing, and orchestrated batch workflows.
  • Build reliable data ingestion systems connecting to provider APIs, webhooks, files, and web scraping sources, with appropriate rate limiting and robust error-handling mechanisms.
  • Implement scheduled workflows and historical data backfills using orchestration tools such as Airflow, Dagster, or Prefect.
  • Develop event-driven processing systems using messaging infrastructure and task queues, including retries, dead-letter queues, and controlled replay capabilities.
  • Store original source payloads in cloud storage and use dbt to transform data into structured raw, standardized, and curated layers.
  • Implement idempotent upserts, data contracts, automated validation, pipeline testing, and monitoring to detect freshness issues and processing failures.
  • Design, maintain, and optimize PostgreSQL schemas and data-loading strategies to support performance, reliability, and scalability.
  • Integrate external provider enrichment and structured data extraction capabilities using Large Language Models (LLMs).
  • Take end-to-end ownership of the platform, covering architectural decisions, implementation, deployment, ongoing maintenance, and production reliability.
Requirements:
  • At least 5 years of experience building and operating production data platforms, particularly those involving multi-source ingestion and end-to-end platform ownership.
  • Production experience with workflow orchestration tools, including managing dependencies, retries, recovery processes, and complex workflows. Airflow is preferred, while Dagster, Prefect, Temporal, or comparable technologies are also relevant.
  • Hands-on production experience with dbt and Google Cloud Platform (GCP).
  • Strong Python programming skills, including experience with asynchronous HTTP requests and concurrent processing.
  • Advanced SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading techniques.
  • Experience designing and operating event-driven processing systems, messaging queues, and reliable data pipelines.
  • Knowledge of data contracts, raw and curated storage architectures, data retention policies, automated testing, and pipeline monitoring.
  • Ability to design resilient ingestion and transformation processes that handle failures, maintain data consistency, and support recovery.
  • Experience building data platforms or substantial pipelines from the ground up is a strong advantage.
  • Familiarity with web scraping, entity resolution, company data enrichment, and LLM-based data extraction is beneficial.
  • Experience working in a small engineering team or startup environment is also an advantage.
Benefits:
  • Annual salary range of USD 100,000–140,000.
  • Fully remote working arrangement within eligible European locations, including Switzerland, the United Kingdom, Norway, Ukraine, Iceland, and Liechtenstein.
  • Opportunity to take end-to-end ownership of a data platform and contribute to its architecture and long-term development.
  • Opportunity to work with modern data engineering technologies, cloud infrastructure, event-driven systems, and LLM-powered data enrichment.
  • Visa sponsorship is not available for this position.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Reltio Data Engineer
Staff Reltio Data Engineer

Jobgether • Germany

Remote
EUR 120,000 - 180,000
100% Remote Work
International Opportunities
Professional Development
+2
Senior Data Scientist – Applied ML & GenAI
Senior Data Scientist – Applied ML & GenAI

Jobgether • Germany

Remote
EUR 90,000 - 130,000
Fully remote
Global client exposure
Senior responsibilities
+1
Senior Software Engineer, Platform
Senior Software Engineer, Platform

Jobgether • Germany

Remote
EUR 110,000 - 150,000
Remote-first
Flexible hours
Laptop provided
+5
Senior Data Engineer (m/f/d)
Senior Data Engineer (m/f/d)

celebrate company • Germany

Hybrid
USD 77,204 - 104,426
Flexible remote work
30 days paid vacation
Internal development opportunities
Senior DevOps Engineer (Azure+AWS)
Senior DevOps Engineer (Azure+AWS)

Jobgether • Germany

Hybrid
EUR 90,000 - 120,000
Remote/Hybrid options
Competitive salary
Career development opportunities
+3
Engineering Manager - Pipelines
Engineering Manager - Pipelines

Jobgether • Germany

Remote
EUR 140,000 - 190,000
Fully remote work
Equity ownership
Technology allowance
+1
Senior Fullstack Engineer (TypeScript, Node.js and Effect.js)
Senior Fullstack Engineer (TypeScript, Node.js and Effect.js)

Jobgether • Germany

Remote
EUR 90,000 - 120,000
Company equity program
25 days annual leave
Carry over up to 10 unused vacation
+9
Senior Data Engineer — Lead Data Platform & Analytics
Senior Data Engineer — Lead Data Platform & Analytics

celebrate company • Germany

Hybrid
USD 77,204 - 104,426
Flexible remote work
30 days paid vacation
Internal development opportunities
Senior Data Engineer
Senior Data Engineer

Ai Futures • München

Hybrid
EUR 80,000 - 110,000
Modern, innovative workplace
Collaborative team culture
Flexible working arrangements
+1
Data QA Engineer
Data QA Engineer

Jobgether • Germany

Remote
EUR 60,000 - 90,000
Fully Remote Work
35 days paid holidays per year
International team of 250+ colleagues
+3