Research Engineer: Build AI Evaluation Environments

mercor.com; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Semi-annual performance bonus

Equity grant vested over 4 years

Up to $15k Relocation bonus

Up to $10k housing bonus

$1.5K monthly meals stipend

Free Equinox membership

$200 monthly laundry reimbursement

$200 monthly personal wellness stipend

Health, Dental, Vision insurance

Job summary

Mercor seeks a senior engineer to help build out high-fidelity RL environments for capability evaluations and training datasets for frontier AI labs. You will transform enterprise data into scalable evals and real-world workflows, contributing to the automation of end-to-end environment production.

You will work across infrastructure, application code, and analytics, delivering data to customers and shaping the future of agentic transformation for diverse enterprises.

Qualifications

• Prior experience shipping environments – contributed to an OSS framework, built environments at previous companies, or worked on agentic evaluations.
• Strong full-stack engineering skills – responsible for everything from infrastructure to app code to analytics.
• Bias to action – focused on shipping evals, not just philosophizing about them.
• Curiosity – eager to understand and dig deep into model behavior and data.
• Sweat the details to make a simulation indistinguishable from the real thing with systems-level thinking to scale quality.

Responsibilities

• Ship models for workflow extraction, classification, and grading.
• Engineer autonomous task refinement processes distilling data taste into pipelines.
• Deliver data to customers and deploy into real engagements.
• Help define the future of agentic transformation for enterprises worldwide.
• Deeply learn about enterprises through building evaluations for all aspects of work.
• Build end-to-end environments for labs and enterprises by platformizing sandbox app clones, loading real data into sandboxes, building prompts from real workflows, and writing verifiers leveraging enterprise expertise and golden outputs.
• Systematize the production of environments to scale throughput while maintaining high-quality worlds and verifiers.

Skills

Full-stack development

OSS contributions

Agentic evaluations

Systems thinking

Education

Bachelor's degree in Computer Science or related field

Tools

Temporal

Modal

Job description

Mercor seeks a senior engineer to help build out high-fidelity RL environments for capability evaluations and training datasets for frontier AI labs. You will transform enterprise data into scalable evals and real-world workflows, contributing to the automation of end-to-end environment production.

You will work across infrastructure, application code, and analytics, delivering data to customers and shaping the future of agentic transformation for diverse enterprises.