Research Scientist — AI Safety Benchmarks & Evaluation

Plato; San Francisco (CA); Sur place; USD 180 000 - 260 000; Plein temps

Résumé du poste

Plato, an applied AI research lab in the San Francisco Bay Area, is hiring an individual contributor researcher to design safety environments and benchmarks. You will establish a new research direction, measure AI R&D capabilities, and work with engineers to turn designs into runnable environments with clear scoring and limits.

You will investigate reward hacking, deception, and other misaligned behavior, owning the research agenda while collaborating with lab partners. Relocation encouraged.

Qualifications

• Recent hands-on work in frontier-model evaluation, benchmarks, or AI safety in the past year or two.
• Meaningful contributions to papers, released benchmarks, or public reports.
• Evidence-based view of frontier models, capabilities, limitations, and likely development.
• Ability to explain what an evaluation measures and its potential gaming risks.
• Ability to turn ambiguous questions into concrete experimental designs with engineers.
• Interest in safety-focused evaluation and potential training-time implications.

Responsabilités

• Design Safety Benchmarks to measure AI research automation and self-improvement.
• Build Safety Environments exposing reward hacking, sabotage, deception, and constraint breaches.
• Hold the Evaluation Bar by examining contamination, grader weaknesses, and task ambiguity.
• Track Frontier Capabilities by running experiments and analyzing trajectories and failures.
• Collaborate with Engineering to turn designs into reproducible environments with scoring and instrumentation.
• Share findings through benchmark releases, reports, and lab partner recommendations.

Connaissances

Frontier-model evaluation

AI safety research

Experimental design

Research communication

Collaboration with engineers

Python programming

Outils

Python

RL frameworks

Description du poste

Plato, an applied AI research lab in the San Francisco Bay Area, is hiring an individual contributor researcher to design safety environments and benchmarks. You will establish a new research direction, measure AI R&D capabilities, and work with engineers to turn designs into runnable environments with clear scoring and limits.

You will investigate reward hacking, deception, and other misaligned behavior, owning the research agenda while collaborating with lab partners. Relocation encouraged.