Legal AI Response Evaluation Lawyer

OpenTrain AI, Inc.; Northern (KY); Hybrid; USD 124,000 - 193,000; Part time

Job summary

OpenTrain AI, Inc. seeks part-time contractors to review written responses generated by AI systems for legal and technical prompts. You will assess clarity, tone, instruction compliance, and analytical quality, and compare responses against standards.

The role requires a US/Canada/UK law degree and native or near-native English. You will record findings on online review platforms, provide detailed rationales, and collaborate asynchronously with other reviewers to maintain consistent evaluation.

Qualifications

• Requires strong analytical review and writing skills.
• Experience with evaluating legal or regulatory texts is preferred.
• Ability to clearly articulate reasoning and identify unsupported claims.

Responsibilities

• Review AI-generated responses for legal/technical prompts.
• Assign quality scores and provide detailed rationales.
• Record findings on online review platforms and maintain consistency with peers.
• Collaborate asynchronously with other reviewers to align standards.

Skills

Analytical review

Quality scoring

Legal writing review

Education

Law degree (US/Canada/UK/IE/Australia/NZ)

Job description

The work

You will review written responses generated by AI systems for legal and technical prompts. Your work will focus on analytical review, not legal advice or representation.

You will compare responses, assign quality scores, and explain each assessment in detailed written rationales. You will record findings in online review platforms and work asynchronously with other reviewers to keep evaluation standards consistent.

• Evaluate clarity, tone, instruction compliance, and analytical quality.
• Identify unsupported claims, overstatements, ambiguity, and factual inaccuracies.
• Check whether legal reasoning is precise, properly qualified, and supported by available facts.
• Assess whether responses fit legal, regulatory, client, or public audiences.
• Manage your assigned deliverables and collaborate with reviewers asynchronously.

What it pays and takes

This is a part-time contractor engagement for applicants with legal writing and analysis experience. The role is listed as entry level, but applicants must meet the education and language requirements below.

• Pay: $90 to $140 per hour in USD.
• Time: 20 or more hours per week.
• Language: First-language English or fully equivalent written fluency.
• Education: A qualifying law degree from a university in the United States, Canada, the United Kingdom, Ireland, Australia, or New Zealand.
• Relevant experience: Drafting briefs, motions, memoranda, legal opinions, or regulatory submissions.
• Writing experience: Reviewing, editing, or supervising legal writing.
• Core strengths: Critical analysis, attention to detail, clear written communication, and the ability to separate supported legal reasoning from confident but unsupported claims.
• Helpful background: Experience with digital document review platforms, annotation tools, model evaluation, or reinforcement learning from human feedback.

About OpenTrain and AI training work

OpenTrain AI is the hiring and contracting organization for this role and helps people build careers in AI training and data labeling. AI training work uses human review to improve how systems understand and produce information, including evaluating written model responses for accuracy, clarity, and usefulness.