Frontier AI Behavior Research Scientist

Transluce; San Francisco, Northern (CA, KY); Hybrid; USD 250,000 - 450,000; Full time

Job summary

Transluce, a fast-moving nonprofit research lab in San Francisco, seeks an AI Behavior Researcher to lead automated evaluations of frontier AI systems, focusing on honesty and alignment. You will design novel evaluations, write analysis code, and collaborate with governance to drive impactful public-policy work.

You will build environment simulators and LLM-based judge pipelines, contributing to scalable, rigorous research while growing with a highly collaborative team and, when needed, visa

Qualifications

• Expertise in quantitative AI evaluation and measurement.
• Experience designing automated AI evaluation methods, such as LLM-as-a-judge systems or multi-turn benchmarks.
• Proficiency in Python to implement analysis and evaluation tooling.
• Meticulous, good experimental design, epistemic self-awareness and transparency.
• Ability to iterate quickly and balance between scrappiness and thoroughness based on impact needs.

Responsibilities

• Develop novel, valid automated evaluations of AI agents' honesty and alignment.
• Write code to implement and run automated evaluations, such as environment simulators or LLM-as-a-judge pipelines.
• Design methods to improve the ecological of automated evaluations, and measure effects as models become more capable.
• Collaborate with governance to deliver high-impact evaluations for public policy.
• Collaborate with scientists and research engineers to productionize best practices in AI behavior evaluation.

Skills

AI evaluation

Python

Experimental design

Communication

Epistemic transparency

Tools

LLM-as-a-judge pipelines

Environment simulators

Job description

Transluce, a fast-moving nonprofit research lab in San Francisco, seeks an AI Behavior Researcher to lead automated evaluations of frontier AI systems, focusing on honesty and alignment. You will design novel evaluations, write analysis code, and collaborate with governance to drive impactful public-policy work.

You will build environment simulators and LLM-based judge pipelines, contributing to scalable, rigorous research while growing with a highly collaborative team and, when needed, visa