Tech Lead, Evals — End-to-End ML Evaluation Platform

Mercor; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Bi-annual performance bonus

Generous equity grant

Relocation bonus

Housing allowance

Meals stipend

Health, Dental, Vision

Wellness reimbursement

Gym membership

Job summary

Mercor is seeking a Tech Lead for the Evals team in San Francisco to own end‑to‑end product development for model evaluation. You will guide architecture, drive APIs and UI, and directly work with labs and enterprises to advance Mercor’s evals platform.

You will tackle grading open‑ended work, maintain reproducibility as tasks change, and scale long agent tasks while expanding into post‑training areas. You’ll mentor engineers and push the platform toward new product areas.

Qualifications

• Bachelor's Degree in Computer Science or a related field.
• 5+ years building and running backend systems in production, with senior or staff level experience.
• Strong full-stack skills in a modern backend language (Python, Go or TypeScript) and a frontend framework such as React.
• A track record of owning a platform or product area end to end, including leading the team that built it.
• Strong product sense that starts from what the user needs.
• Experience building data-heavy products: data models, APIs and views over large volumes of results.
• Excellent written and verbal communication with engineers, researchers and customers.
• Comfort with ambiguity and a habit of shipping iteratively.

Responsibilities

• Own the evals flywheel and its direction: Set architecture and roadmap, and make tradeoff calls in design reviews.
• Lead the product customers use to evaluate models: Ship APIs, data models and UI for labs and enterprises to run evals and compare models.
• Power APEX launches: Build systems that run every new frontier model on APEX and publish correct results to the public leaderboard.
• Explain why models fail: Build failure analysis linking failure patterns to exact steps that went wrong.
• Set the quality bar for evals: Calibrate graders and decide when a grader or agent change is ready to ship.
• Keep production healthy: Ensure the product stays reliable and compute costs stay in check.
• Work directly with customers: Lead pilots with labs and enterprises and feed learnings back to the roadmap.
• Grow the team: Hire and mentor engineers, and manage team members.

Skills

Python

Go

TypeScript

React

Platform ownership

Data-heavy products

Communication

Ambiguity tolerance

Education

Bachelor's Degree in Computer Science or related field

Tools

Backend systems

Job description

Mercor is seeking a Tech Lead for the Evals team in San Francisco to own end‑to‑end product development for model evaluation. You will guide architecture, drive APIs and UI, and directly work with labs and enterprises to advance Mercor’s evals platform.

You will tackle grading open‑ended work, maintain reproducibility as tasks change, and scale long agent tasks while expanding into post‑training areas. You’ll mentor engineers and push the platform toward new product areas.