Founding AI Evaluation Engineer — Shape Agent Testing

Platform Recruitment; San Francisco (CA); On-site; USD 180,000 - 300,000; Full time

Job summary

Platform Recruitment in San Francisco is seeking its first engineer for an early-stage, venture-backed AI startup. You will help enterprises understand how AI agents perform, where they fail, and how to improve them, shaping a new category from the ground up, working with a seasoned founder from the AI evaluation world.

You\'ll design scoring methods, build test scenarios, and decide how much testing is enough, ensuring results are statistically sound and trusted by customers, while contributing

Qualifications

• Experience building evaluation systems and graders.
• Background applying scientific measurement to ML outputs.
• Early-stage founder or early hire experience.

Responsibilities

• Design scoring methods for agent outputs on complex tasks.
• Build realistic test scenarios and simulated workflows.
• Ensure testing results are statistically sound and trusted by customers.
• Enable core product improvements from customer deployments.

Skills

Strong engineering skills

Measurement mindset

Founding experience

Tools

RL environments

Synthetic data

Evaluation systems

Job description

Platform Recruitment in San Francisco is seeking its first engineer for an early-stage, venture-backed AI startup. You will help enterprises understand how AI agents perform, where they fail, and how to improve them, shaping a new category from the ground up, working with a seasoned founder from the AI evaluation world.

You\'ll design scoring methods, build test scenarios, and decide how much testing is enough, ensuring results are statistically sound and trusted by customers, while contributing