AI Benchmarking Fellow: Open Datasets & Evaluations

snorkelai; United States; Hybrid; USD 117,000 - 227,000; Full time

Benefits offered by this job

Hourly compensation

In-person time in SF or NY offices

Research co-advisor access

Community events access

Job summary

snorkelAI is seeking a Research Fellow for an AI data lab to help teams build datasets and environments behind frontier AI capabilities. The fellowship targets PhD students and postdocs who want to translate questions into new evaluation methods.

Fellows retain their academic affiliation while spending at least one day per week with the research team, including in-person time in San Francisco or New York offices where applicable.

Qualifications

• Research background sufficient to design and build open benchmarks, datasets, or evaluation environments in your field.
• Demonstrated ability to turn research ideas into working code, datasets, tools, or other usable research artifacts.
• Ability to collaborate with specialists outside your own field and convert their judgment into a clear specification.
• Capacity to scope a project to a public release within the fellowship term.

Responsibilities

• Scope and lead a new benchmark, dataset, or evaluation environment in a domain where current evaluations fall short.
• Define the task specification, rubrics, and reference answers alongside credentialed domain experts, and run the expert review process.
• Release the research artifact publicly as an open-source academic deliverable with you as lead author.
• Join weekly research meetings to share progress, receive feedback, and contribute to other researchers' projects.

Skills

Research background

Coding / datasets development

Cross-disciplinary collaboration

Job description

snorkelAI is seeking a Research Fellow for an AI data lab to help teams build datasets and environments behind frontier AI capabilities. The fellowship targets PhD students and postdocs who want to translate questions into new evaluation methods.

Fellows retain their academic affiliation while spending at least one day per week with the research team, including in-person time in San Francisco or New York offices where applicable.