Human Data Quality Analyst, AI Business

prolific; Deutschland; Hybrid; EUR 42.000 - 58.000; Vollzeit

Benefits dieser Stelle

Hybrid working model

Equity

Bonus potential

Zusammenfassung

Prolific in Germany is seeking an early-career analyst to join the human-data side of frontier AI, focusing on data quality for AI training and evaluation. The role blends statistical analysis with in-depth review of human annotations to identify failure modes and drive improvements across programmes.

You will run quality measurements, investigate shifts, help build the data-quality pipeline with rule-based, LLM-assisted, and automated validations, and translate findings into clear reports for

Qualifikationen

• 1–2 years of practical experience analysing real data in an analytical, quality, research, or data-focused role.
• Working proficiency in SQL and Python for queries, analysis, and interpreting results.

Aufgaben

• Run day-to-day quality measurement across live human-data programmes using statistical analysis and hands-on review.
• Investigate quality shifts, quantify impact, and distinguish individual errors from systemic issues in guidance, task design, or tooling.
• Help build the data-quality pipeline alongside quality engineers, including rule-based, LLM-assisted, and automated validations.
• Translate complex analysis into clear reporting and visualisations for technical and non-technical stakeholders.
• Support new programmes from design through launch, contributing to rubrics, guidelines, gold sets, and calibration exercises.
• Use findings from live programmes to improve annotation design, guidance, training, and overall quality processes.

Kenntnisse

SQL

Python

Statistics

Data analysis

Tools

Dashboards

Jobbeschreibung

Role overview

Join the human-data side of frontier AI as an early-career analyst focused on data quality for AI training and evaluation. The work goes beyond traditional QA, combining statistical analysis with close review of human-generated annotations to identify failure modes and drive improvements across programmes. It is an analytical role designed for someone looking to grow quickly while contributing to a fast-evolving AI infrastructure platform.

Responsibilities

• Run day-to-day quality measurement across live human-data programmes using statistical analysis and hands-on review.
• Investigate quality shifts, quantify impact, and distinguish individual errors from systemic issues in guidance, task design, or tooling.
• Help build the data-quality pipeline alongside quality engineers, including rule-based, LLM-assisted, and automated validations.
• Translate complex analysis into clear reporting and visualisations for technical and non-technical stakeholders.
• Support new programmes from design through launch, contributing to rubrics, guidelines, gold sets, and calibration exercises.
• Use findings from live programmes to improve annotation design, guidance, training, and overall quality processes.

Requirements

• One to two years of practical experience analysing real data in an analytical, quality, research, or data-focused role.
• Working proficiency in SQL and Python for queries, analysis, and interpreting results.
• Applied understanding of statistical concepts such as sampling, distributions, and variability.
• Ability to turn evidence into clear accounts with concrete examples and practical recommendations.
• Careful, consistent assessment of detailed work combined with pattern recognition across datasets.
• Comfort testing approaches, documenting reasoning, and asking for input when the right method is unclear.

Nice to have

• Experience with data produced or judged by people, such as annotation, labelling, research coding, or survey research, including agreement or gold-set metrics.
• Background in psychology, behavioural science, linguistics, or a related field.
• Familiarity with how human data supports supervised fine-tuning, RLHF, preference data, and evaluation benchmarks.
• Experience creating dashboards or clear data visualisations.

Benefits and work setup

• Hybrid working model with access to a unique human-data platform used by frontier AI researchers.
• Compensation packages include base salary, equity, and benefits; many roles also include a bonus or commission element.