Founding Engineer, Environments + Data

Wwshemi; San Francisco (CA); Sur place; USD 200 000 - 300 000; Plein temps

Avantages offerts par ce poste

Founding equity

Visa sponsorship

Résumé du poste

Wwshemi is seeking a founding engineer to design synthetic company environments and develop evaluation frameworks for AI agents. The role blends research with production readiness to prove agent capabilities in real-world company tasks.

You will build benchmarks, graders, and simulation environments, ensuring high fidelity tools and scalable evaluation methods. Visa sponsorship is available, and equity is part of the package.

Qualifications

• Hands-on research experience in synthetic data pipelines, benchmarks, evaluations, or model post-training.
• Experience building environments or benchmarks used to train or evaluate agents at AI labs.
• Experience implementing graders or evaluation systems for AI agents.

Responsabilités

• Design synthetic company environments with realistic data for agents to use over time.
• Build graders and evaluation systems for advanced agents and cases with limited expert oversight.
• Develop benchmarks that correlate with real-world production performance.
• Identify and mitigate evaluation failure modes such as reward hacking and benchmark gaming.
• Simulate software tools (inboxes, CRMs) to support large-scale agent interactions.

Connaissances

Synthetic data pipelines

Benchmarks design

Evaluation systems

AI agent evaluation

Research experience

Formation

Degree from selective university

Founder program participation

Outils

CRMs

Email inbox simulation

Description du poste

This listing's locations On-site - San Francisco, California, United States

Employment Full-time

Workplace On site

Compensation $200000–300000 per year

Build evaluation environments and benchmarks that measure whether AI agents can perform real-world company work and meet or exceed human performance. This founding engineering role combines agent evaluation research with production readiness, helping establish reliable evidence of agent capability.

What You'll Do

• Design synthetic company environments with realistic email, records, and files for agents to use over extended periods.
• Build graders and evaluation systems capable of assessing advanced agents, including cases where human reviewers may lack the relevant expertise.
• Develop benchmarks that correlate with real-world production performance.
• Find and address evaluation failure modes, including reward hacking and benchmark gaming.
• Simulate software tools such as inboxes and CRMs with enough fidelity to support agent interactions at scale.

What We're Looking For

• Hands-on research experience in synthetic data pipelines, benchmarks, evaluations, or model post-training; this is a research-oriented role, not a pure infrastructure engineering position.
• Experience building environments or benchmarks used to train or evaluate agents at a frontier AI lab, and implementing graders or evaluation systems for AI agents.
• At least 1 to 2 years at a fast-growing or AI-focused startup, plus mid-level or senior engineering or research experience.
• Experience simulating software systems for agent tool use, identifying benchmark flaws, and detecting and fixing reward hacking in evaluation systems.
• A degree from a highly selective university or an equivalent credential, such as founding a company or participating in a recognized founder program.
• High-level competition experience in mathematics, programming, debate, or games is a plus.

Base salary is $200,000 to $300,000 annually, with up to $350,000 for exceptional candidates. Additional commission-style bonus of up to $200,000 or more is tied to the value of delivered environments. Meaningful founding equity is included. Visa sponsorship is available.