Senior Data Engineer - AI Training Data Platform

Prosus

Amsterdam

Hybrid

EUR 90,000 - 130,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Relocation support
Remote work arrangements
MacBook Pro
Competitive compensation

Job summary

Prosus is seeking a senior data engineer to build and operate the LCM training-data platform across our global ecommerce and AI initiatives. You will own pipelines for acquisition, processing, synthetic data generation, quality evaluation and dataset versioning, collaborating with scientists and ML engineers to translate data into reproducible training runs.

You'll work in our Amsterdam-based AI team, with opportunities to work remotely for top candidates, and contribute to governance, data

Qualifications

  • At least five years of relevant data engineering experience in production pipelines.
  • Strong Python and SQL skills with tested, maintainable code.
  • Hands-on experience with distributed processing using Spark, Ray, Beam or a comparable framework.
  • Experience with cloud storage, workflow orchestration and operating pipelines through changes and backfills.
  • Knowledge of data quality, versioning and lineage; ability to translate research needs into reliable implementations.

Responsibilities

  • Build acquisition and processing pipelines for portfolio data and relevant public and licensed sources, including connectors and content extraction.
  • Develop the shared platform for inspecting sources, configuring processing and publishing training datasets. Operate distributed workflows, monitor reliability and cost, and implement access controls and retention requirements.
  • Work with scientists, engineers and the relevant teams on data provenance, permitted use across companies and privacy-aware handling of customer histories.
  • Implement dataset versioning and lineage across source snapshots, processing code and dataset configurations, so releases can be reproduced and traced to the training runs that used them.
  • Build quality checks for completeness, freshness, duplication, extraction accuracy and coverage. Combine automated checks with sampled review to investigate problems that aggregate metrics can miss.
  • Provide tools for source tagging, sampling and weighting, and maintain evaluation datasets with temporal splits and overlap checks. Work with scientists on controlled comparisons to establish whether data changes improve model quality.
  • Build synthetic data pipelines supporting pre- and post-training, including instruction examples, preference data and agent trajectories. Implement generation, validation, scoring and filtering. Work with the research team to measure how the generated data affects model performance, and manage generation throughput and cost.
  • Build agents that turn catalogue sources into training datasets: propose tasks based on available data and target model capabilities, configure processing and generation, curate outputs and investigate quality failures. Keep agent actions traceable and validate proposed changes before dataset release.

Skills

Data engineering
Python
SQL
Distributed processing
Cloud workflows
Data quality & lineage
Versioning
LLM datasets
Agents for data tasks

Tools

Spark
Ray
Beam
Databricks
EMR

Job description

Prosus is seeking a senior data engineer to build and operate the LCM training-data platform across our global ecommerce and AI initiatives. You will own pipelines for acquisition, processing, synthetic data generation, quality evaluation and dataset versioning, collaborating with scientists and ML engineers to translate data into reproducible training runs.

You'll work in our Amsterdam-based AI team, with opportunities to work remotely for top candidates, and contribute to governance, data

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer, Training Data Platform
Senior Data Engineer, Training Data Platform

Prosus • Amsterdam

Hybrid
EUR 90,000 - 130,000
Relocation support
Remote work arrangements
MacBook Pro
+1
Senior Data & AI Platforms Engineer — Hybrid (Utrecht/Home)
Senior Data & AI Platforms Engineer — Hybrid (Utrecht/Home)

Apply Recruitment • Utrecht

Hybrid
EUR 90,000 - 130,000
Real Impact on millions of users
Modern MLOps stack
Balanced hybrid model (Utrecht/Home)
Senior Data Platform Engineer: AI & MLOps
Senior Data Platform Engineer: AI & MLOps

WCC Group • Utrecht

Hybrid
EUR 90,000 - 140,000
Senior Data & AI Engineer: Build AI Platforms & Mentor Team
Senior Data & AI Engineer: Build AI Platforms & Mentor Team

Datacation • Netherlands

Remote
EUR 85,000 - 125,000
Senior Applied AI Scientist - Research to Production
Senior Applied AI Scientist - Research to Production

Arbeitnow • Amsterdam

On-site
EUR 140,000 - 210,000
Relocation support
Competitive compensation
Amsterdam office / AI House
Senior Applied AI Scientist — Lead Next-Gen Commerce AI
Senior Applied AI Scientist — Lead Next-Gen Commerce AI

Prosus • Amsterdam

Hybrid
EUR 120,000 - 170,000
Amsterdam office base
Relocation support
Remote work arrangements
+1
Senior AI Scientist, Commerce Foundation Models (Remote)
Senior AI Scientist, Commerce Foundation Models (Remote)

Vacaturebank • Amsterdam

Hybrid
EUR 120,000 - 180,000
Relocation support
MacBook Pro
Remote work options
Senior Data Engineering & ML Leader (Hybrid, Amsterdam)
Senior Data Engineering & ML Leader (Hybrid, Amsterdam)

Relocate.me • Amsterdam

Hybrid
EUR 170,000 - 230,000
Advanced relocation package
Adaptation tips
Flight ticket
+3
Senior Data Engineer
Senior Data Engineer

EPAM Systems • Netherlands

Hybrid
EUR 90,000 - 140,000
Senior Applied AI Scientist — Lead Foundation Model R&D
Senior Applied AI Scientist — Lead Foundation Model R&D

Vacaturebank • Amsterdam

Hybrid
EUR 140,000 - 200,000
Relocation support
Amsterdam office base
Remote work arrangements
+2