LLM Evaluation Scientist: Frontier Benchmarking

Anyone AI; Buenos Aires; Presencial; ARS 178.909.546 - 268.364.320; Jornada completa

Descripción de la vacante

Anyone AI Labs is seeking a Research Scientist to lead evaluations and benchmarking for large language models. You will design frontier-grade evaluation methods and benchmarks across reasoning, coding, agents, tool use, and multimodal tasks, grounded in expert-verified truth and validated across multiple models.

You will own the study lifecycle from framing questions to publishing results, with opportunities to contribute to public benchmarks and papers at top venues, while collaborating with

Formación

• Research background in ML evaluation or benchmarking.
• Deep LLM/frontier-model benchmarking expertise, especially code-model and agentic evaluation.
• Fluency with measurement concepts: construct validity, rubrics, headroom, contamination.

Responsabilidades

• Evaluation research: design new benchmark targets and evaluation methods.
• Benchmark development: build evaluation packages with ground truth and multi-model results.
• Experts: recruit and review a pool across coding, agents, and reasoning.
• Lab relationships: act as technical contact for labs with CEO support.
• Delivery & dissemination: translate lab requests into pilots and publish benchmarks/papers.

Conocimientos

ML evaluation

Benchmarking

English fluency

Descripción del empleo

Anyone AI Labs is seeking a Research Scientist to lead evaluations and benchmarking for large language models. You will design frontier-grade evaluation methods and benchmarks across reasoning, coding, agents, tool use, and multimodal tasks, grounded in expert-verified truth and validated across multiple models.

You will own the study lifecycle from framing questions to publishing results, with opportunities to contribute to public benchmarks and papers at top venues, while collaborating with