Legal Domain Expert – AI Evaluator & SME (Freelance)

Uber AI Solutions; United States; On-site; USD 95,000 - 130,000; Full time

Job summary

Uber AI Solutions is seeking an early-career Legal Domain Expert to apply practical training in legal research, citation analysis, and statutory interpretation to evaluate and train next-generation legal LLMs.

You will perform citation reviews, draft realistic scenarios from statutes, regulations, court opinions and contracts, and develop rigorous rubrics and golden answers, ensuring accuracy and compliance across jurisdictions.

Qualifications

• JD/LLM or Bachelor's degree with 3-5 years post-graduate experience in legal research, paralegal work, litigation support, or law firm practice.
• Experience with primary source document analysis and rigorous citation checking (Bluebook).
• Strong command of legal drafting and citation standards.

Responsibilities

• Audit primary and secondary authority citations for accuracy and detect hallucinated or misquoted authorities.
• Conceptualize and draft realistic, multi-step legal scenarios grounded in statutes, regulations, court opinions, contracts and pleadings.
• Develop objective evaluation rubrics and publish golden responses adhering to Bluebook/local citation standards.

Skills

Legal Research

Citation Verification

AI Familiarity

Education

JD/LLM

Bachelor's in Legal Studies

Tools

Westlaw

LexisNexis

Casetext

Job description

Uber AI Solutions connects qualified legal professionals with client-led Generative AI research initiatives. As an early-career Legal Domain Expert, you will leverage your practical training in legal research, citation analysis, and statutory interpretation to evaluate and train next-generation legal LLMs.

In this role, you will apply hands-on legal reasoning to conduct rigorous citation reviews, construct fine-grained evaluation rubrics, analyze complex agent trajectories, and audit AI outputs for hallucinations and procedural errors.

• Citation & Authority Auditing: Perform rigorous verification of primary and secondary authority citations produced by legal AI models to detect hallucinated case law, misquoted statutory provisions, and invalid citations.
• Prompt & Scenario Engineering: Conceptualize and draft realistic, multi-step legal scenarios grounded in primary source documents (statutes, regulations, court opinions, contracts, and filed pleadings).
• Rubric Development & Ground Truth Creation: Build objective evaluation rubrics and author publication-grade "golden" responses adhering to precise Bluebook/local citation and formal legal drafting standards.
• Failure Mode & Trajectory Analysis: Red-team model outputs to systematically document failure modes, including secondary authority confusion, jurisdiction over-generalization, and errors in multi-document synthesis or deadline computations.
• Workflow Compliance: Complete onboarding calibration, run self-directed quality assurance checks, and incorporate peer reviewer feedback into final submissions.

Qualifications & Experience

Education & Experience

• Juris Doctor (JD), Master of Laws (LLM), or Bachelor's degree in legal studies with 3-5 years of post-graduate experience in legal research, paralegal work, litigation support, or law firm practice.

Core Competencies

• Legal Research & Verification: Strong command of industry research tools (Westlaw, LexisNexis, Casetext), primary source document analysis, and rigorous citation checking (e.g., The Bluebook).
• Jurisdictional Tracks: Priority interest in US Corporate, US Securities, US Labor, or Commercial Litigation.
• AI Familiarity: Experience or strong interest in Generative AI evaluation, prompt testing, RLHF, data annotation, or golden-answer authoring is a plus.