Research Fellow, Open AI Benchmarks & Datasets

Snorkel AI; United States; Hybrid; USD 117,000 - 227,000; Full time

Benefits offered by this job

Research co-advisor

Compute support

Open-source artifact

Team & events

Communications support

Job summary

Snorkel AI is seeking a Research Fellow to build open benchmarks and evaluation environments that advance AI progress. You’ll own projects from the initial research question to public release, with a co-advisor from Snorkel and access to expert input and compute resources.

You’ll maintain your academic affiliation while spending at least one day a week with Snorkel’s team, including in-person work in San Francisco or New York where applicable.

Qualifications

• Strong research background with ability to design open benchmarks, datasets, or environments.
• Proven ability to turn research ideas into working code, datasets, tools, or other usable artifacts.
• Ability to collaborate with domain experts and translate their judgment into specifications.

Responsibilities

• Scope and lead a new benchmark, dataset, or environment in a domain where current evaluations fall short
• Define the task specification, rubrics, and reference answers with domain experts, and run the expert review process
• Release the research artifact publicly, with you as lead author
• Join weekly research meetings to share progress, receive feedback, and contribute to other researchers’ projects

Skills

Academic research

Code and data tooling

collaboration with domain experts

Education

PhD student or postdoc

Job description

Snorkel AI is seeking a Research Fellow to build open benchmarks and evaluation environments that advance AI progress. You’ll own projects from the initial research question to public release, with a co-advisor from Snorkel and access to expert input and compute resources.

You’ll maintain your academic affiliation while spending at least one day a week with Snorkel’s team, including in-person work in San Francisco or New York where applicable.