Staff Engineer, AI Benchmarking & Evaluation

Handshake; New York (NY); On-site; USD 270,000 - 350,000; Full time

Benefits offered by this job

Ownership: Equity in a fast-growing公司

Financial Wellness: 401(k) match, comp

Family Support: Paid parental leave,育儿

Wellbeing: Medical, dental, vision, $/

Growth: Learning stipend, development

Office: Commuting support, free lunch,

Time Off: Flexible PTO, 15 holidays +

Connection: Team outings & referral

Job summary

Handshake is hiring a Member of Technical Staff to define how frontier coding agents are benchmarked and improved. You will collaborate with top AI researchers and engineers to develop benchmarks, verifier systems, and data-quality techniques for coding applications.

Early team members influence technical direction, culture, and open-source software. Location: San Francisco preferred; exceptional candidates in other locations are encouraged to apply.

Qualifications

• Passion for building software with agents.
• Ability to work in an ambiguous, fast-moving environment with substantial ownership.
• Experience with coding benchmarks or published AI for software/code-generation work is a plus.

Responsibilities

• Create/publish Coding benchmarks that challenge state-of-the-art frontier agents.
• Understand/shape the frontier of coding agents and publish studies of findings.
• Help the world’s AI labs improve their models for coding.
• Publicly contribute through benchmarks, open-source tools, research, and technical writing.

Skills

Coding benchmarks

Agent harnesses

OSS contributions

GitHub profile

Job description

Handshake is hiring a Member of Technical Staff to define how frontier coding agents are benchmarked and improved. You will collaborate with top AI researchers and engineers to develop benchmarks, verifier systems, and data-quality techniques for coding applications.

Early team members influence technical direction, culture, and open-source software. Location: San Francisco preferred; exceptional candidates in other locations are encouraged to apply.