Senior Legal AI Trainer & Evaluation Expert (Bay Area)

Mercor; San Francisco (CA); Hybrid; USD 83,000 - 138,000; Full time

Benefits offered by this job

Referral reward up to $1,600

On-site work in Bay Area

Job summary

Mercor is seeking a senior legal domain expert to collaborate with a leading AI lab's research and program management teams to improve how frontier AI models reason about real legal work. You will review the quality of legal knowledge tasks, write instruction specifications and golden solutions, and help build benchmarks that demonstrate genuine model improvement.

This is a full-time W-2 role with Cincinnatus LLC, based in the Bay Area, California.

Qualifications

• JD from an accredited law school.
• 5+ years of substantive post-qualification legal practice at a reputable institution.
• Genuine specialization in at least one substantive practice area (e.g., corporate, litigation, regulatory, IP, employment, tax).
• Active admission to at least one U.S. state bar, in good standing.
• Hands-on experience with large language models in professional work; ability to judge a well-reasoned answer from a plausible-sounding wrong one.
• Able to commit reliably to 40 hours per week for an initial engagement of 6 months.
• Living in the Bay Area, California, and able to work on-site with the client's team multiple days per week.

Responsibilities

• Data QA and reviews: Vet the quality of legal knowledge work tasks and model outputs; spot gaps and weak reasoning.
• Write instruction specs and golden datasets to define correct solutions for legal problems.
• Design benchmarks and domain-specific evaluation sets with the research team.
• Calibrate standards with client researchers and translate tacit legal judgment into explicit criteria.

Education

Juris Doctor (JD) from an accredited law school

Job description

Mercor is seeking a senior legal domain expert to collaborate with a leading AI lab's research and program management teams to improve how frontier AI models reason about real legal work. You will review the quality of legal knowledge tasks, write instruction specifications and golden solutions, and help build benchmarks that demonstrate genuine model improvement.

This is a full-time W-2 role with Cincinnatus LLC, based in the Bay Area, California.