Senior Legal AI Benchmark & QA Architect

Obsidian; Chicago (IL); Hybrid; USD 180,000 - 260,000; Full time

Job summary

Obsidian is seeking a senior legal domain expert to join its GenAI team, working with research and program management to improve how frontier AI models reason about real legal work. This is a full-time W-2 role through Cincinnatus LLC, with on-site Bay Area work and potential placement at a leading AI lab.

You will review quality, write instruction specs and golden solutions, and design benchmarks. Requires JD, 5+ years post-qualification, US bar, and hands-on experience with LLMs.

Qualifications

• JD from an accredited law school; rank preferred.

Responsibilities

• Data QA and reviews: Vet the quality of legal knowledge work tasks and model outputs.
• Instruction specs and golden datasets: Write high-quality instruction specs, produce golden solutions to legal problems, and define new legal tasks.
• Benchmarks and domain depth: Design challenging legal tasks and evaluation sets, and help build legal-specific skills and tools.
• Calibration: Translate tacit legal judgment into explicit, teachable criteria and align with client standards.

Skills

Legal expertise

AI fluency

Data QA

Instruction design

Benchmark design

Calibration

Written communication

40h/week availability

Education

Juris Doctor (JD) from an accredited law school

Active U.S. state bar admission

Tools

Legal research databases

AI model evaluation tooling

Job description

Obsidian is seeking a senior legal domain expert to join its GenAI team, working with research and program management to improve how frontier AI models reason about real legal work. This is a full-time W-2 role through Cincinnatus LLC, with on-site Bay Area work and potential placement at a leading AI lab.

You will review quality, write instruction specs and golden solutions, and design benchmarks. Requires JD, 5+ years post-qualification, US bar, and hands-on experience with LLMs.