Technical Staff, Data - London

Reflection AI; London; On-site; GBP 40,000 - 65,000; Full time

Job summary

A leading company in AI research is seeking a data-focused team member to develop and analyze data collection techniques. This interdisciplinary role blends engineering, research, and communication to enhance data quality and diversity in model training. You will collaborate with top researchers and engineers, building scalable data pipelines and conducting thorough data analysis.

Benefits

Work at the forefront of AI research

Collaboration with world-class researchers

Competitive compensation and benefits

Qualifications

• Experience building data processing pipelines at scale.
• Proficiency in Python and familiarity with other languages like Go or TypeScript.
• Detail-oriented with strong organizational skills.

Responsibilities

• Develop techniques for data collection and analysis.
• Design experiments with machine learning researchers.
• Manage human annotators for data collection efforts.

Skills

Software engineering

Data analysis

Python

Communication

Analytical thinking

Job description

Role Overview:

Data quality and diversity is the foundation for training the best agents in any domain. As a member of the Data Team at Reflection, you will play a pivotal role in shaping how we collect and analyze human, synthetic, and internet data. This is an interdisciplinary role that primarily requires engineering, research, and communication skills, along with a sharp attention to detail and willingness to “roll up your sleeves” and look at the data.

Key Responsibilities

1. Experiment and Benchmark Design

• Develop techniques for collecting, augmenting, filtering, or synthesizing training and evaluation data using creativity and analytical thinking

• Design experiments, in collaboration with machine learning researchers, to assess the impact of different datasets on model performance

• When required, manage human annotators working on data collection efforts – this could include tracking payments and hours, training annotators, and providing technical support, feedback, and quality control

2. Qualitative and Quantitative Data Analysis

• Analyze collected data, e.g. coding tasks, both qualitatively and quantitatively

• Evaluate model behavior to identify its strengths and weaknesses

• Clearly communicate findings with machine learning research and product teams

3. Data Engineering

• Design, implement, and optimize scalable data pipelines to support reinforcement learning and supervised finetuning

• Leverage LLMs to perform data filtering, cleaning, and augmentation

Qualifications:

• Software engineering background with experience building data processing pipelines at scale, particularly with LLM integration

• Proficiency in Python or other programming languages (Go, TypeScript, etc.)

• Detail-oriented and analytical, with the ability to conduct careful qualitative and quantitative data analysis.

• Excellent organizational and communication skills to collaborate closely with cross-functional teams and manage human data operations

• Experience with machine learning, reinforcement learning, and LLMs is a plus, but not strictly required.

What We Offer:

• The opportunity to work at the forefront of AI research and data collection for training cutting-edge models.

• Collaboration with a team of world-class researchers and engineers from top AI labs and companies.

• Competitive compensation and benefits, with opportunities for professional growth.