LLM Data Quality & Evaluation Specialist

Cohere; Greater London; Hybrid; GBP 70,000 - 90,000; Full time

Job summary

A leading AI company is seeking a Member of Technical Staff for Data Analysis and Evaluation in London. In this role, you will ensure the quality and performance of large language models by designing data collection tasks and evaluating datasets. You will collaborate with cross-functional teams and require strong software engineering and statistical skills. The company offers an inclusive culture and benefits including remote flexibility, health coverage, and six weeks vacation.

Benefits

Open and inclusive culture

Weekly lunch stipend

Health and dental benefits

Parental Leave top-up

Personal enrichment benefits

Remote flexibility

6 weeks of vacation

Qualifications

• Extremely strong software engineering skills.
• Strong expertise in data collection tasks.
• Experience with human annotators.
• Statistical skills for evaluating experiments.
• Hands-on experience training large language models.

Responsibilities

• Design data collection tasks and ensure quality.
• Develop statistical methods for dataset evaluation.
• Analyse robustness of ML systems.
• Collaborate to improve model performance.
• Conduct experiments for model evaluation.

Skills

Software engineering

Statistical analysis

Data collection design

Communication

Machine learning frameworks

Education

PhD or equivalent experience in AI/Data Science

Tools

Python

PyTorch

TensorFlow

JAX

Job description

A leading AI company is seeking a Member of Technical Staff for Data Analysis and Evaluation in London. In this role, you will ensure the quality and performance of large language models by designing data collection tasks and evaluating datasets. You will collaborate with cross-functional teams and require strong software engineering and statistical skills. The company offers an inclusive culture and benefits including remote flexibility, health coverage, and six weeks vacation.