Remote Frontier AI Evaluation Architect

We Love AI Jobs; San Francisco (CA); Remote; USD 200,000 - 350,000; Full time

Job summary

Handshake is hiring a Member of Technical Staff, Evals to shape how frontier AI systems are measured, understood, and improved. You will design evaluation frameworks and benchmarks for language models, agents, and multimodal systems, while developing reward models, verifiers, and data-quality tooling that enable reliable, high-signal feedback.

You will collaborate with researchers and engineers to publish results, contribute to open-source projects, and influence our technical direction as an

Qualifications

• PhD in ML/AI or equivalent research experience.
• Publications at top AI/ML venues (NeurIPS/ICML/ICLR).
• Strong Python skills and experience building scalable software.

Responsibilities

• Design evaluation frameworks and benchmarks for frontier LLMs, agents, and multimodal models.
• Develop reward models, verifiers, graders, and data-quality systems.

Skills

Python

LLM evaluation

AI agent evaluation

Benchmark design

Reward models

Programmatic verifiers

Reinforcement-learning environments

Scalable software

Education

PhD or equivalent research experience

Tools

GitHub

OSS contributions

Job description

Handshake is hiring a Member of Technical Staff, Evals to shape how frontier AI systems are measured, understood, and improved. You will design evaluation frameworks and benchmarks for language models, agents, and multimodal systems, while developing reward models, verifiers, and data-quality tooling that enable reliable, high-signal feedback.

You will collaborate with researchers and engineers to publish results, contribute to open-source projects, and influence our technical direction as an