Founding Engineer

Platform Recruitment; San Francisco (CA); On-site; USD 180,000 - 300,000; Full time

Job summary

Platform Recruitment in San Francisco is seeking its first engineer for an early-stage, venture-backed AI startup. You will help enterprises understand how AI agents perform, where they fail, and how to improve them, shaping a new category from the ground up, working with a seasoned founder from the AI evaluation world.

You\'ll design scoring methods, build test scenarios, and decide how much testing is enough, ensuring results are statistically sound and trusted by customers, while contributing

Qualifications

• Experience building evaluation systems and graders.
• Background applying scientific measurement to ML outputs.
• Early-stage founder or early hire experience.

Responsibilities

• Design scoring methods for agent outputs on complex tasks.
• Build realistic test scenarios and simulated workflows.
• Ensure testing results are statistically sound and trusted by customers.
• Enable core product improvements from customer deployments.

Skills

Strong engineering skills

Measurement mindset

Founding experience

Tools

RL environments

Synthetic data

Evaluation systems

Job description

Founding Engineer, AI Agent Evaluation | San Francisco (Onsite) | Up to $300k base + significant equity

An early-stage, venture-backed AI startup in San Francisco is hiring its first engineer. The company helps enterprises understand whether their AI agents are actually doing the job: how well they perform, where they break and how to improve them.

This is a chance to shape a new category from the ground up, working side by side with an experienced founder from the AI evaluation world.

What you'll work on:

• Designing ways to score agent output on complex business tasks where there's no single right answer
• Building realistic test scenarios and simulated workflows that reflect how agents are used in practice
• Deciding how much testing is enough, and making sure results are statistically sound and trusted by customers
• Taking approaches that work in individual customer deployments and building them into the core product

You'll likely have:

• Built evaluation systems, benchmarks, graders, reward models, RL environments or synthetic data for LLMs or agents
• Strong engineering skills, paired with a scientific approach to measurement
• Early-stage experience as a founder, founding engineer or early hire
• Around 2 to 6 years of experience

Bonus points for:

post-training experience (RLHF, SFT or similar), domain experience in areas like finance, legal or healthcare, or a background in statistics or published research on evaluation.