Staff Evaluations Scientist for Open-Weight Models

Reflection AI; Greater London; On-site; GBP 75,000 - 110,000; Full time

Benefits offered by this job

Top-tier compensation

Comprehensive health insurance

Fully paid parental leave

Paid time off and relocation support

Daily lunch and dinner provided

Job summary

An AI research company in Greater London is seeking a candidate for a role focused on evaluating AI models. The position involves conducting comparative analysis, developing evaluation frameworks, and collaborating with various teams to improve model features. Ideal candidates will have strong statistical analysis skills and familiarity with LLM evaluation methodologies. The role is designed for those excited to thrive in a fast-paced startup environment and contribute to groundbreaking AI developments.

Qualifications

• Strong statistical analysis and experimental design skills to rigorously measure model improvements.
• Familiarity with LLM evaluation methodologies such as static benchmarks and human preference evals.
• High agency and thrive in a fast-paced startup environment; bias for impact over process.

Responsibilities

• Conduct critical comparative analysis to advance understanding of model capabilities.
• Build and refine evaluation systems and processes for tight feedback loops.
• Develop generalizable evaluation frameworks for reasoning, alignment, and usefulness.

Skills

Statistical analysis

Experimental design

LLM evaluation methodologies

Collaboration

Job description

An AI research company in Greater London is seeking a candidate for a role focused on evaluating AI models. The position involves conducting comparative analysis, developing evaluation frameworks, and collaborating with various teams to improve model features. Ideal candidates will have strong statistical analysis skills and familiarity with LLM evaluation methodologies. The role is designed for those excited to thrive in a fast-paced startup environment and contribute to groundbreaking AI developments.