Tech Lead, AI Evals Platform

mercor.com; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Bi-annual bonus

Equity grant

Relocation bonus up to $15k

Housing stipend

Meal stipend

Health insurance

Job summary

Mercor seeks a Tech Lead for the Evals team to own the product end to end, from architecture to hands-on coding. You will shape the evals flywheel, grade open-ended work, and drive scalable pipelines for frontier models.

You’ll partner with labs and enterprises, lead a growing team of engineers, and translate failures into actionable data that improves models. This is a hands-on leadership role in a fast‑growing AI product company with global reach.

Qualifications

• Bachelor's degree in CS or a related field.
• 5+ years building backend systems in production.
• Strong full-stack skills in Python, Go, or TypeScript and a frontend framework such as React.
• A track record of owning a platform or product area end to end, including leading the team that built it.
• Strong product sense that starts from what the user needs.
• Experience building data-heavy products: data models, APIs and views over large volumes of results.
• Excellent written and verbal communication with engineers, researchers and customers.
• Comfort with ambiguity and a habit of shipping iteratively.

Responsibilities

• Own the evals flywheel and its direction: Set the architecture and roadmap, and make the tradeoff calls in design reviews.
• Lead the product customers use to evaluate models: Ship the APIs, data models and UI that labs and enterprises use to run evals and compare models.
• Power APEX launches: Build the systems that run every new frontier model on APEX and publish correct, defensible results to the public leaderboard.
• Explain why models fail: Build failure analysis that links each failure pattern to the exact steps that went wrong.
• Set the quality bar for evals: Calibrate graders against expert judgment and decide when a grader or agent change is ready to ship.
• Keep production healthy: Make sure the product stays reliable and its compute costs stay in check.
• Work directly with customers: Lead pilots with labs and enterprises and bring what you learn back into the roadmap.
• Grow the team: Hire and mentor engineers, and manage the people on the team.

Skills

Backend systems

Full-stack dev

Python/Go/TypeScript

React

Platform ownership

Data modeling

APIs

Communication

Ambiguity tolerance

Education

Bachelors in CS or related

Job description

Mercor seeks a Tech Lead for the Evals team to own the product end to end, from architecture to hands-on coding. You will shape the evals flywheel, grade open-ended work, and drive scalable pipelines for frontier models.

You’ll partner with labs and enterprises, lead a growing team of engineers, and translate failures into actionable data that improves models. This is a hands-on leadership role in a fast‑growing AI product company with global reach.