Research Fellow: Open Benchmarks & AI Evaluation

Snorkel AI; San Francisco (CA); Hybrid; USD 117,000 - 227,000; Full time

Benefits offered by this job

Research advisor

Compute resources

Open-source artifact

Community events

Job summary

Snorkel AI is seeking a Research Fellow to build open benchmarks and evaluation environments that help shape how AI progress is measured. You will own the project from the initial research question through public release, with a Snorkel researcher serving as co-advisor and access to compute and domain experts.

You’ll retain your academic affiliation while spending at least one day a week with Snorkel’s research team, including in person in our San Francisco or New York offices where applicable.

Qualifications

• PhD student or postdoc in a relevant field who values open benchmarks
• Ability to turn research ideas into working code, datasets, tools, or artifacts
• Comfort collaborating with domain experts outside your field

Responsibilities

• Scope and lead a new benchmark, dataset, or environment in a domain where current evaluations fall short
• Define the task specification, rubrics, and reference answers with domain experts from Snorkel's network, and run the expert review process
• Release the research artifact publicly, with you as lead author
• Join weekly research meetings to share progress, receive feedback, and contribute to other researchers’ projects

Job description

Snorkel AI is seeking a Research Fellow to build open benchmarks and evaluation environments that help shape how AI progress is measured. You will own the project from the initial research question through public release, with a Snorkel researcher serving as co-advisor and access to compute and domain experts.

You’ll retain your academic affiliation while spending at least one day a week with Snorkel’s research team, including in person in our San Francisco or New York offices where applicable.