Staff Scientist, Frontier AI Evaluations & Benchmarks

Handshake; Mountain View (CA); On-site; USD 190,000 - 260,000; Full time

Benefits offered by this job

Health Insurance

401k match

Sabbatical

Learning & development stipend

Parental leave

Flexible time off

Volunteer time off

Job summary

Handshake seeks a Member of Technical Staff, Evals to define how frontier AI systems are measured, understood, and improved. You will partner with AI researchers, domain experts, and customers to develop benchmarks, reward systems, verifiers, and data-quality techniques.

Early members influence our technical direction, culture, and open-source software. We value demonstrated research capability and a builder’s mindset over titles or degree.

Qualifications

• PhD in ML/AI, CS, data science, or related fields or equivalent industry research experience.
• Publications at top AI/ML venues (NeurIPS, ICML, ICLR, COLM).
• Strong Python skills and experience building scalable software.
• Builders who ship benchmarks, datasets, or software with an OSS portfolio.
• Deep knowledge of frontier AI benchmarks, eval techniques, and agent harnesses.
• Active conference participation and public open-source contributions.

Responsibilities

• Design and build evaluation frameworks, benchmarks, and methodologies for frontier LLMs, AI agents, multimodal models, and RL environments.
• Develop reward models, programmatic verifiers, graders, and data-quality techniques.
• Research what makes evaluations representative, reliable, and resistant to shortcutting.
• Build systems for high-quality human data with task design and data attribution.
• Run fast iteration loops: prototype, evaluate, interpret results, diagnose failures, and improve benchmarks.
• Publicly contribute to the field through benchmarks, open-source tools, and research writing.

Skills

Python

Research

Publications

Ownership

Agent development

Education

PhD in ML/AI or related field

Tools

GitHub

Open-source software

Job description

Handshake seeks a Member of Technical Staff, Evals to define how frontier AI systems are measured, understood, and improved. You will partner with AI researchers, domain experts, and customers to develop benchmarks, reward systems, verifiers, and data-quality techniques.

Early members influence our technical direction, culture, and open-source software. We value demonstrated research capability and a builder’s mindset over titles or degree.