Staff Software Engineer, AI Safety & Evaluations

Anthropic Limited; San Francisco (CA); Presencial; USD 320.000 - 485.000; Jornada completa

Descripción de la vacante

Anthropic Limited is seeking a Staff+ Software Engineer for Safeguards Evals in San Francisco. You will design experiments to evaluate model safety, build datasets representing real misuse, and ship evaluation methods into training and deployment pipelines.

You will work at the intersection of ML research and engineering to improve how we evaluate model behavior and agentic systems. You will own evaluation methodology, create production-ready tooling, and collaborate with policy and enforcement

Formación

• Experience building and maintaining data pipelines.
• Working with LLMs and understanding their capabilities and failure modes.
• Strong data analysis skills to draw reliable insights from large datasets.
• Ability to move from research prototyping to production-quality code.
• Translate ambiguous problems into concrete, testable experiments.
• Care deeply about AI safety and real-world impact.

Responsabilidades

• Evaluation research and methodology: design and run experiments to improve evaluation quality; simulate real user behavior; validate grading accuracy.
• Agentic investigation evals: build and own evaluation harness for long-horizon agent, measure end-to-end performance, drive improvements, construct RL environments.
• Datasets grounded in real harm: create high-quality eval datasets covering real-world misuse; collaborate with Policy and Enforcement.
• Productionization and tooling: ship research into evaluation and release pipelines; build tools for policy experts to author and run evaluations.

Conocimientos

Data pipelines

LLMs

Data analysis

Prototyping to production

Problem translation

AI safety

Descripción del empleo

Anthropic Limited is seeking a Staff+ Software Engineer for Safeguards Evals in San Francisco. You will design experiments to evaluate model safety, build datasets representing real misuse, and ship evaluation methods into training and deployment pipelines.

You will work at the intersection of ML research and engineering to improve how we evaluate model behavior and agentic systems. You will own evaluation methodology, create production-ready tooling, and collaborate with policy and enforcement