Evals Research Scientist / Engineer

COL Limited; City Of London; On-site; GBP 100,000 - 200,000; Full time

Job summary

A leading research organization in London seeks Research Scientists and Engineers to conduct safety evaluations and collaborate with top-tier AI labs like OpenAI and Google DeepMind. Ideal candidates should have a strong foundation in empirical research, excellent communication skills, and software engineering experience in Python. The role is full-time, with competitive salary and extensive benefits including unlimited vacation and professional development budget.

Benefits

Flexible work hours

Unlimited vacation

Paid work trips

Yearly professional development budget

Qualifications

• Experience in empirical research related to AI control and evaluations.
• Strong scientific writing and communication skills.
• Comprehensive experience in LLM steering.

Responsibilities

• Conduct safety evaluations and develop theoretical frameworks.
• Collaborate with frontier labs on pre-deployment evaluations.
• Automate the evals pipeline.

Skills

Experience in empirical research

Excellent scientific writing

Experience in LLM steering

Software engineering skills

Tools

Python

Inspect (evals framework)

Job description

Application deadline: We're accepting applications until 31 October 2025. We encourage early submissions and will start interviews in early October.

ABOUT THE OPPORTUNITY

We’re looking for Research Scientists and Research Engineers who are excited to work on safety evaluations, the science of scheming, or control/monitoring for frontier models.

YOU WILL HAVE THE OPPORTUNITY TO

• Work with frontier labs like OpenAI, Anthropic, and Google DeepMind, by running pre-deployment evaluations and collaborating closely on mitigations, see e.g. our work on anti-scheming or OpenAI’s o1-preview system card and Anthropics’s Opus 4 and Sonnet 4 system card.
• Build evaluations for scheming-related properties (such as deceptive reasoning, sabotage, and deception tendencies). See our conceptual work on scheming, e.g. evaluation-based safety cases for scheming or how scheming could arise.
• Work on the science of scheming, e.g. by studying model organisms or real-world examples of scheming in detail. Our goal is to develop a much better theoretical understanding of why models scheme and which components of training and deployment cause it.
• Work on automating the entire evals pipeline. We aim to automate substantial parts of evals ideation, generation, running and analysis.
• Design and evaluate AI control protocols. Since agents have longer and longer time-horizons, we're shifting more effort to deployment-time monitoring and other control methods.
• Note: We are not hiring for interpretability roles.

KEY REQUIREMENTS

• We don’t require a formal background or industry experience and welcome self-taught candidates.
• Experience in empirical research related to scheming, AI control and evaluations and a scientific mindset: You have designed and executed experiments. You can identify alternative explanations for findings and test alternative hypotheses to avoid overinterpreting results. This experience can come from academia, industry, or independent research.
• Track record of excellent scientific writing and communication: You can understand and communicate complex technical concepts to our target audience and synthesize scientific results into coherent narratives.
• Comprehensive experience in Large Language Model (LLM) steering and the supporting Data Science and Data Engineering skills. LLM steering can take many different forms, such as: a) prompting, b) LM agents and scaffolding, c) fluent LLM usage and integration into your own workflows, d) experience with supervised fine-tuning, e) experience with RL on LLMs.
• Software engineering skills: Our entire stack uses Python. We're looking for candidates with strong software engineering experience.
• (Bonus) We have recently switched to Inspect as our primary evals framework, and we value experience with it.
• Depending on your preferred role and how these characteristics weigh up, we can offer either a RS or RE role.

We want to emphasize that people who feel they don’t fulfill all of these characteristics but think they would be a good fit for the position, nonetheless, are strongly encouraged to apply. We believe that excellent candidates can come from a variety of backgrounds and are excited to give you opportunities to shine.

LOGISTICS

• Start Date: Target of 2-3 months after the first interview.
• Time Allocation: Full-time
• Location: The office is in London, and the building is shared with the London Initiative for Safe AI (LISA) offices. This is an in-person role. In rare situations, we may consider partially remote arrangements on a case-by-case basis.
• Work Visas: We can sponsor UK visas

BENEFITS

• Salary: 100k - 200k GBP (~135k - 270k USD)
• Flexible work hours and schedule
• Unlimited vacation
• Unlimited sick leave
• Lunch, dinner, and snacks are provided for all employees on workdays
• Paid work trips, including staff retreats, business trips, and relevant conferences
• A yearly $1,000 (USD) professional development budget

ABOUT APOLLO RESEARCH

The rapid rise in AI capabilities offer tremendous opportunities, but also present significant risks.

At Apollo Research, we’re primarily concerned with risks from Loss of Control, i.e. risks coming from the model itself rather than e.g. humans misusing the AI. We’re particularly concerned with deceptive alignment / scheming, a phenomenon where a model appears to be aligned but is, in fact, misaligned and capable of evading human oversight. We work on the detection of scheming (e.g., building evaluations), the science of scheming (e.g., model organisms), and scheming mitigations (e.g., anti-scheming and control). We closely work with multiple frontier AI companies, e.g. to test their models before deployment or collaborate on scheming mitigations.

At Apollo, we aim for a culture that emphasizes truth-seeking, being goal-oriented, giving and receiving constructive feedback, and being friendly and helpful.

ABOUT THE TEAM

The current evals team consists of several researchers. You will mostly work with the evals team, but you will likely sometimes interact with the governance team to translate technical knowledge into concrete recommendations.

Equality Statement: Apollo Research is an Equal Opportunity Employer. We value diversity and are committed to providing equal opportunities to all, regardless of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, or sexual orientation.

How to apply: Please complete the application form with your CV. The provision of a cover letter is optional but not necessary. Please also feel free to share links to relevant work samples.

About the interview process: Our multi-stage process includes a screening interview, a take-home test (approx. 2.5 hours), 3 technical interviews, and a final interview with Marius (CEO). The technical interviews will be closely related to tasks the candidate would do on the job.