AI Safety Benchmarks Lead — Ship Frontiers & Roadmaps

Alice (Formerly ActiveFence); New York (NY); Presencial; USD 190 000 - 240 000; Tempo integral

Resumo da oferta

Alice (Formerly ActiveFence) in New York, NY seeks a leader to shape harms research benchmarks. You will own taxonomy, harness, and release cycles, coordinating researchers and freelancers while aligning with the CTO and research leads.

You will ship benchmarks every 2–3 weeks, uphold the quality bar, manage timelines, and guide roadmaps with the CTO, pod, and research leads. Expect conference travel and collaboration with AI labs.

Qualificações

• PhD or Masters in computer science, machine learning or a related field, or equivalent depth from industry research.
• 3+ years building and running safety or security evaluations for language models in production, at an AI lab, a model provider, or a safety and security research organisation.
• 5+ relevant research publications in AI safety and security including lead author on at least 2 of them.
• Strong engineer. Evaluation harnesses, distributed inference, vLLM, reading a codebase and fixing it.
• You can build a taxonomy, not only score against one.
• You can direct a researcher and two freelancers without managing them formally.
• Strong English, written and spoken. This role runs on internal communication across time zones.
• Curiosity about the harms themselves. You will be learning a new subject every three weeks.

Responsabilidades

• Ship a benchmark cadence: a benchmark every two to three weeks with taxonomy-driven evals.
• Own the quality bar: ensure verifiers, rubrics, and taxonomy are sound; screen evals and ensure novelty.
• Run the process: coordinate timelines and ad-hoc direct two or three freelancers as needed.
• Set the roadmap with the forum: monthly input with CTO, pod, and research leads to update release plans.
• Stay ahead of the curve: maintain ecosystem ties and engage labs weekly; travel to conferences.

Conhecimentos

Strong English communication

Experiment design & evaluation

Codebase reading & debugging

Research leadership

Formação académica

PhD or Masters in CS/ML or related field

5+ AI safety/security publications (lead author on 2)

Descrição da oferta de emprego

Alice (Formerly ActiveFence) in New York, NY seeks a leader to shape harms research benchmarks. You will own taxonomy, harness, and release cycles, coordinating researchers and freelancers while aligning with the CTO and research leads.

You will ship benchmarks every 2–3 weeks, uphold the quality bar, manage timelines, and guide roadmaps with the CTO, pod, and research leads. Expect conference travel and collaboration with AI labs.