Senior Data Processing Engineer

European Tech Recruit; Berlin; Hybrid; EUR 70,000 - 110,000; Full time

Job summary

European Tech Recruit in Berlin is seeking a Senior ML Data Processing Engineer to design and optimise large-scale data pipelines for ML training. You will work at the intersection of data engineering, NLP and ML, turning raw web-scale data into high-quality training datasets.

You will implement data quality controls, deduplication, privacy safeguards, and reproducibility measures while collaborating with researchers and engineers to ensure reliable data for advanced model development.

Qualifications

• PhD in a relevant technical discipline.
• 3+ years in data processing, ML engineering, NLP or related field.
• Experience with massive unstructured text datasets and distributed frameworks (Spark, Ray, Flink).
• Strong Python skills for production-grade pipelines.
• Experience in data curation, deduplication, filtering, quality datasets.
• Understanding of data privacy, PII removal and reproducibility.
• Experience with workflow tools (Airflow, Prefect, Dagster).
• Experience with LLMs, Docker/Kubernetes, web-scale data acquisition and licensing.

Responsibilities

• Build and optimise distributed pipelines for large-scale ML training data.
• Implement data curation, deduplication, filtering and metadata extraction.
• Develop ML-based evaluators and automated data-quality systems.
• Establish dataset versioning, lineage and reproducibility.
• Monitor pipeline performance and data quality; identify bottlenecks.
• Collaborate with Research, Engineering and Governance on privacy and licensing.

Skills

Python

Data pipelines

ML engineering

NLP

Distributed processing

Data privacy

Education

PhD in a relevant technical discipline

Tools

Apache Spark

Ray

Flink

Docker/Kubernetes

vLLM

SGLang

Airflow

Prefect

Dagster

Job description

Berlin, Germany | Hybrid - 3 days onsite per week

Join a highly technical AI research and engineering organisation building the data foundations for next-generation AI models. As a Senior ML Data Processing Engineer, you’ll develop and optimise large-scale data pipelines that transform raw, web-scale data into high-quality training datasets.

You’ll work at the intersection of data engineering, machine learning and NLP, improving data quality through automated filtering, deduplication, ML-based evaluation and privacy controls. Working closely with researchers and engineers, you’ll help ensure training datasets are reliable, reproducible and suitable for advanced model development.

Key Responsibilities

• Build and optimise distributed pipelines for processing large-scale, unstructured datasets for ML training.
• Implement data curation and quality controls, including deduplication, filtering, PII removal, toxicity detection and metadata extraction.
• Develop automated data-quality systems using ML classifiers, heuristics and LLM-based evaluators.
• Establish dataset versioning, lineage and reproducibility, including mechanisms to detect data leakage and evaluation contamination.
• Monitor pipeline performance and data quality, identifying bottlenecks, regressions and opportunities to improve efficiency.
• Collaborate with Research, Engineering and Governance teams on dataset requirements, privacy, licensing and compliance.

Required Experience & Skills

• PhD in a relevant technical discipline.
• At least 3 years of relevant experience in data processing, ML engineering, NLP or a closely related field.
• Experience working with massive, unstructured text datasets and distributed processing frameworks such as Apache Spark, Ray or Flink.
• Strong Python skills and experience building production-grade data-processing pipelines.
• Practical experience with data curation, deduplication, filtering, dataset quality or large-scale dataset construction.
• Understanding of data privacy, PII removal, content filtering, evaluation contamination and dataset reproducibility.
• Experience with workflow orchestration tools such as Airflow, Prefect or Dagster.
• Experience with LLM training or fine-tuning, LLM inference tools such as vLLM or SGLang, Docker/Kubernetes, web-scale data acquisition, data licensing or open-source NLP projects.

By applying to this role you understand that we may collect your personal data and store and process it on our systems. For more information please see our Privacy Notice (https://eu-recruit.com/about-us/privacy-notice/)