Founding Engineer: AI Environments & Benchmarks

Clera; San Francisco (CA); On-site; USD 150,000 - 300,000; Full time

Benefits offered by this job

Founding equity

Visa sponsorship available

Job summary

Clera in San Francisco, CA, is seeking a founding engineer to build evaluation environments and benchmarks for AI agents performing company work. This role sits at the intersection of agent evaluation and production readiness, with direct influence on how agent competence is assessed.

You design synthetic company environments, graders, and benchmarks; address evaluation failures like reward hacking; simulate tools such as inboxes and CRMs; join a startup with equity and visa sponsorship options.

Qualifications

• Experience building environments or benchmarks used by frontier AI labs to train or evaluate agents.
• Hands‑on experience designing and implementing AI agent graders or evaluation systems.
• Experience simulating software systems that can withstand agent tool use, and building synthetic environments for production‑like evaluation.
• A track record of identifying flaws in widely used benchmarks and detecting and fixing reward hacking in graders.
• Startup or early‑stage company experience, strong competitive problem‑solving experience in areas such as math, programming, debate, or games, and a collaborative approach to ambiguous, fast‑paced work.
• Strong conviction in the potential of AI agents to perform computer-based work.

Responsibilities

• Design synthetic company environments with realistic email, records, and files for agents to use over extended periods.
• Build robust graders and evaluation systems that can assess agents, including when their performance may exceed human reviewers’ expertise.
• Develop benchmarks that reflect real-world production performance.
• Find and address evaluation failures, including reward hacking and benchmark gaming.
• Simulate software tools such as inboxes and CRMs to support realistic agent interactions at scale.

Skills

Environment benchmarks

AI graders

Tool simulation

Reward hacking detection

Startup experience

Competitive problem-solving

Tools

CRM simulation

Job description

Clera in San Francisco, CA, is seeking a founding engineer to build evaluation environments and benchmarks for AI agents performing company work. This role sits at the intersection of agent evaluation and production readiness, with direct influence on how agent competence is assessed.

You design synthetic company environments, graders, and benchmarks; address evaluation failures like reward hacking; simulate tools such as inboxes and CRMs; join a startup with equity and visa sponsorship options.