Senior Data Engineer (m/f/d)

GlassDollar; Berlin; On-site; EUR 85,000 - 120,000; Full time

Job summary

GlassDollar is seeking a senior data engineer to design and ship robust data pipelines for ingestion, scraping, enrichment and entity resolution. You will own the data model, establish observability, and build LLM-based classification for unstructured sources.

You will work with engineers and product to deliver measurable quality data, signals and events, with a strong focus on GDPR compliance and scalable architecture in a Berlin-based workspace.

Qualifications

• Five plus years in data engineering or backend with data focus.
• Built and run production data pipelines end to end with orchestration and monitoring.
• Fluent Python for scrapers and enrichment services; production-ready.
• Strong SQL on a production Postgres database.
• Hands-on with LLMs for extraction and classification.
• Entity resolution and deduplication at scale.
• Data quality testing (dbt tests, GE or similar).
• Knowledge of GDPR for company/people data.
• Based in Berlin, or willing to relocate.

Responsibilities

• Own ingestion and pipelines for vendor feeds, APIs, and scraping jobs with monitoring and alerts.
• Define and maintain the schema and data dictionary; manage migrations and Hasura layer.
• Implement entity resolution across sources with measurable precision/recall.
• Build and tune LLM extraction/classification pipelines with cost/latency focus.
• Develop signals pipelines for events like funding, acquisitions, headcount.
• Ensure data quality, observability, tests, and measurement across the platform.
• Evaluate data sources and vendors, advising on quality and cost per datapoint.

Skills

5+ years in data engineering

Production data pipelines

Python for scrapers/enrichment

SQL on Postgres

LLM extraction/classification

Entity resolution

Data quality testing

GDPR knowledge

Based in Berlin / relocation

Tools

Airflow

Dagster

Prefect

Python

PostgreSQL

dbt

Docker

AWS

GCP

Job description

TL;DR

You build and run the pipelines behind GlassDollar's company data: ingestion, scraping, enrichment, entity resolution, LLM extraction and signal detection – from first prototype to reliable production. You sit in the product team, report to our Product Owner and work closely with our engineers.

Our data has grown faster than our engineering around it. Feeds run, but nothing tells us when they stop. Most relationship records aren't classified yet, duplicates slip through, and some signals users ask for, like headcount changes, have no reliable live source. Making this robust, observable and fast is the job.

Who uses your work

• People in vclOS: Search, lists, projects, PoC tracking, recommendations and weekly company updates.
• Our AI agents & search: Clean fields with provenance, confidence and freshness, measured on our search benchmark.
• API and MCP partners: Stable schemas and fields to build on.
• Marketing: Directory, rankings and reports through a quarterly data release.

What we're looking for

What you own

• Ingestion and pipelines: vendor feeds, APIs and scraping jobs – scheduled, idempotent, monitored, with backfills and alerts when a feed stops or drifts.
• Schema and data model: the Postgres schema and data dictionary as the source of truth, including migrations and the Hasura layer on top.
• Entity resolution: matching and deduplicating companies across sources, with measurable precision and recall.
• LLM extraction and classification: pipelines that turn unstructured sources into structured records, evaluated on labelled samples and tuned for cost per record.
• Signals: funding, acquisitions, shutdowns, case studies and headcount as event pipelines with change detection and correct event dates.
• Data quality and observability: automated tests and metrics for coverage, accuracy, freshness and completeness.
• Data sources: you test vendors, APIs and scraping partners on real samples and give a clear recommendation on quality and cost per datapoint.

What you'll do

Your first 90 days

By day 90: every feed is monitored and alerting, relationship data is classified with measured precision, entity resolution has a baseline, 3–5 new datapoints are live in production, and our search benchmark has moved against a target you set in month 1.

Month 1: baseline and observability

• Inventory every pipeline, vendor feed and scraping job; give each one an owner, runbook, monitoring and alerting.
• Data dictionary v1 for the top 25 datapoints, with source, lineage and quality numbers.
• Automated quality checks (coverage, freshness, nulls, duplicates) running on every load.
• Quick win: Scale partnership data point collection.

Month 2: corporate–technology relationships

• Model relationship types in the schema: mention, partnership, PoC, implementation, investment, acquisition.
• Build an LLM classification pipeline for existing and incoming records; report precision on a labelled sample and cost per record.
• Set a recall and lag baseline, then improve the relationship pipeline against it.
• Integrate a live acquisition source and keep acquired companies out of recommendations.

Month 3: new datapoints, entity resolution and agents

• Run 6–8 time‑boxed datapoint experiments, each ending in ship, kill or park. Ship 3–5 to production.
• Entity resolution v2: measurable matching across sources, with a review queue for uncertain cases.
• Add provenance, confidence and freshness to the top 25 fields, and make customer data (lists, PoCs, notes) usable by agents with proper permissions.
• Document the data platform: architecture, runbooks and on‑call handover.

Requirements

Who you are

A senior data engineer who ships. You care about reliability and clean data models, and you understand what the data is used for. You'd rather put a rough datapoint live this week than a perfect one next quarter.

Must-haves

• 5+ years in data engineering (or backend engineering with a data focus), ideally with company, market or B2B data at a data business, marketplace or SaaS company.
• You've built and run production data pipelines end to end: orchestration (e.g. Airflow, Dagster or Prefect), monitoring, alerting and backfills.
• Fluent Python: scrapers and enrichment services (requests or Playwright, pandas or polars), tested and production‑ready.
• Strong SQL on a production Postgres database: schema design, migrations, indexing and query performance.
• Hands‑on with LLMs for extraction and classification, measured on a labelled sample, with an eye on cost and latency.
• Practical entity resolution and deduplication at scale.
• Data quality testing (e.g. dbt tests, Great Expectations or your own checks).
• Working knowledge of GDPR for company and people data.
• Based in Berlin, or willing to relocate.

Nice to have

• dbt, Docker and a major cloud (AWS or GCP); infrastructure as code.
• Event or signal pipelines: change detection, event dating, alerting.
• Vector search or embeddings for matching and retrieval.
• GraphQL / Hasura.
• German at reading level: DACH registers are a big part of our upside.
• Working with data from providers such as PitchBook, Crunchbase, Dealroom or company registers.