Remote Legal Rubric Auditor — Licensed Attorney

Confidential

United States

Remote

USD 83,000 - 165,000

Part time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Confidential is seeking licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing questions, verify model answers against controlling statutes, regulations, and case law, and refine grading rubrics for accuracy and clarity.

This short-term, fully remote engagement requires U.S. residency and strong legal research, writing, and analytical skills across multiple practice areas, including Administrative,

Qualifications

  • JD and active U.S. bar admission in good standing required.
  • 5+ years of legal practice with substantive experience in one or more practice areas listed.
  • Strong research, analytical, and writing abilities essential.
  • High attention to detail and ability to apply consistent judgment.

Responsibilities

  • Audit grading rubrics for accuracy, completeness, consistency, and clarity.
  • Verify ground-truth answers against statutes, regulations, and case law.
  • Identify legal errors, outdated authority, missing issues, and ambiguous criteria.
  • Rewrite or correct ground-truth answers and rubric items as needed.
  • Provide concise, structured written feedback on each item reviewed.

Skills

5+ years of legal practice
Strong legal research, analysis, and -
High attention to detail and judgment

Education

Juris Doctor (JD) and active U.S. bar admission in good standing

Job description

Confidential is seeking licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing questions, verify model answers against controlling statutes, regulations, and case law, and refine grading rubrics for accuracy and clarity.

This short-term, fully remote engagement requires U.S. residency and strong legal research, writing, and analytical skills across multiple practice areas, including Administrative,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Legal Rubric & Ground-Truth Auditor
Remote Legal Rubric & Ground-Truth Auditor

Mercor • United States

Remote
USD 83,000 - 124,000
Remote Legal Rubric & Ground-Truth Auditor (JD, 5+ yrs)
Remote Legal Rubric & Ground-Truth Auditor (JD, 5+ yrs)

Benture • United States

Remote
USD 165,000 - 193,000
Legal Rubric & Ground Truth Auditor (Licensed Attorney)
Legal Rubric & Ground Truth Auditor (Licensed Attorney)

Confidential • United States

Remote
USD 83,000 - 165,000
Legal Rubric & Ground Truth Auditor (Licensed Attorney)
Legal Rubric & Ground Truth Auditor (Licensed Attorney)

Mercor • United States

Remote
USD 83,000 - 124,000
Legal Rubric & Ground Truth Auditor (Attorney) · Mercor Mercor · $120-140/hr · remote in US · 3d ago $120-140/hr remote in US 3d ago
Legal Rubric & Ground Truth Auditor (Attorney) · Mercor Mercor · $120-140/hr · remote in US · 3d ago $120-140/hr remote in US 3d ago

Benture • United States

Remote
USD 165,000 - 193,000
Remote US Attorney Evaluator for AI Legal Tasks Rubrics
Remote US Attorney Evaluator for AI Legal Tasks Rubrics

Vettedlance • United States

Remote
USD 138,000 - 220,000
Remote AI Legal Evaluator & Rubric Architect (Attorney)
Remote AI Legal Evaluator & Rubric Architect (Attorney)

Consilio LLC • United States

Remote
USD 62,000 - 76,000
Remote work
Part-time hours
ASAP start
Legal & Compliance Specialist – Rubric Design
Legal & Compliance Specialist – Rubric Design

SuperAnnotate AI, Inc. • United States

Remote
USD 152,000 - 179,000
Remote Attorney: Legal AI Quality Auditor
Remote Attorney: Legal AI Quality Auditor

LOD group • United States

Remote
USD 103,000 - 110,000
Remote Litigation Attorney — AI Legal Tech Researcher
Remote Litigation Attorney — AI Legal Tech Researcher

Remotebridge • United States

Remote
USD 83,000 - 131,000