AI Safety Research Scientist — Benchmarks & Evaluation

raydar; San Francisco (CA); On-site; USD 250,000 - 450,000; Full time

Benefits offered by this job

Health, dental and vision coverage

Job summary

Raydar, an applied research company in San Francisco, is seeking a Research Scientist to design and run safety evaluations for advanced AI models, building benchmarks and test environments that probe model behavior under pressure and help improve training processes.

You will collaborate with engineers to turn research designs into reproducible evaluations, publish benchmark releases and reports, and share findings with research partners.

Qualifications

• Must have hands-on experience with post-training and reinforcement learning.
• Demonstrated safety work on large-scale models within the last two years.
• Knowledge of frontier model capabilities and directions in the field.
• Publication record in peer-reviewed ML conferences or established evaluation datasets.

Responsibilities

• Set up standardized tests for model safety evaluations with clear baselines and success criteria.
• Build test environments surfacing reward gaming, sabotages and deviations from intended limits.
• Run experiments on current models, track capability changes over time and analyze trajectories.
• Collaborate with engineers to produce reproducible environments with clear scoring and documented limitations.

Skills

Post-training RL

Safety work on large-scale models

Frontier models knowledge

Academic publication track record

Education

PhD in CS or related field OR Bachelor's/Master with substantial experience

Job description

Raydar, an applied research company in San Francisco, is seeking a Research Scientist to design and run safety evaluations for advanced AI models, building benchmarks and test environments that probe model behavior under pressure and help improve training processes.

You will collaborate with engineers to turn research designs into reproducible evaluations, publish benchmark releases and reports, and share findings with research partners.