Frontier AI Evaluation Scientist — Equity & Growth

AfterQuery; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Health Insurance

401(k) with Employer Match

Daily Meals: UberEats stipend

Monthly Wellness Stipend

Job summary

AfterQuery is hiring Research Scientists to design and publish rigorous evaluations for frontier AI systems. The role spans agentic, coding, and safety evaluations, across healthcare, STEM, finance, and related fields.

You will own evaluation development end to end and collaborate across disciplines to turn important capability gaps into rigorous public research. PhD in a related field is preferred, and candidates should have a track record of publishing research.

Qualifications

• Proven track record of publishing benchmarks or research papers.
• Excellent scientific writing and clear technical communication.
• Commitment to experimental rigor including baselines, ablations, and statistical validity.
• Ability to scope ambiguous evaluation questions into reproducible public releases.
• Experience with agentic, coding, and safety evaluations or applied ML in expert domains.

Responsibilities

• Lead end-to-end design and validation of frontier AI benchmarks.
• Collaborate with researchers and domain experts to cover high-priority domains.
• Analyze model capabilities and failure modes using rigorous experimental methods.
• Build reproducible evaluation systems, harnesses, graders, and infrastructure.
• Post-train models and measure resulting performance gains; communicate results via reports and papers.

Skills

Benchmark research

Technical writing

Experimental design

Ambiguity handling

Agentic/safety evals

Education

PhD preferred

Job description

AfterQuery is hiring Research Scientists to design and publish rigorous evaluations for frontier AI systems. The role spans agentic, coding, and safety evaluations, across healthcare, STEM, finance, and related fields.

You will own evaluation development end to end and collaborate across disciplines to turn important capability gaps into rigorous public research. PhD in a related field is preferred, and candidates should have a track record of publishing research.