AI Benchmark Researcher: Design Tasks to Probe AI

Braintrust; Puerto Rico; In loco; Tempo pieno

Descrizione del lavoro

Braintrust seeks experts to identify complex problems in their field and translate them into Linux-based tasks for testing advanced AI agents. You will design challenges evaluated against multiple models, ensuring difficulty and novelty.

Applicants should have 10+ years in technical domains, publications, and be comfortable using a Linux terminal, with the posting noting residency in the United States.

Competenze

• Master’s degree or higher required; candidates with a bachelor’s degree plus 10+ years experience may be considered.
• At least 10 years of professional experience in a technical domain is required.
• Expertise in coding, machine learning, systems, cybersecurity or hardware.
• At least one academic or professional publication is required.
• Practical comfort working in a Linux terminal.
• Currently based in the United States.

Mansioni

• Identify difficult, meaningful problems in your area of expertise and translate them into tasks.
• Design challenges tested against three leading AI models with three attempts per model.
• Prove tasks are difficult enough that models mostly fail.

Conoscenze

Coding expertise

Machine learning

Systems

Cybersecurity

Hardware

Formazione

Master’s degree or higher

Bachelor’s degree + 10 years experience may be considered

Strumenti

Linux terminal

Descrizione del lavoro

Braintrust seeks experts to identify complex problems in their field and translate them into Linux-based tasks for testing advanced AI agents. You will design challenges evaluated against multiple models, ensuring difficulty and novelty.

Applicants should have 10+ years in technical domains, publications, and be comfortable using a Linux terminal, with the posting noting residency in the United States.