Remote Legal & Compliance SME: AI Evaluation

Joinkeywork; United States; A distancia; USD 55.000 - 96.000; A tiempo parcial

Descripción de la vacante

SME Careers, a subsidiary of SuperAnnotate, is seeking a Legal & Compliance SME for an hourly remote contractor role. You will create realistic scenarios, evaluate AI-generated content, and design rigorous rubrics for professional analysis.

You will translate nuanced legal judgment into structured criteria covering issues, standards, risk, jurisdiction, and reasoning. This is a remote, international engagement with emphasis on accuracy and professional rigor in AI training data workflows.

Formación

• Must hold JD, LLB, LLM, or equivalent law degree.
• Strong professional qualification or bar admission preferred for attorney-focused roles.
• Excellent English proficiency (C1+).
• 3+ years in law, regulatory compliance, legal research, policy, risk, privacy, or related field.
• Ability to create challenging scenarios requiring issue spotting, interpretation, and judgment.
• Ability to design rubrics defining issues, standards, reasoning steps, exceptions, risks, and omissions.

Responsabilidades

• Create legal and compliance tasks: develop realistic, difficult scenarios requiring expert analysis and professional judgment.
• Design detailed rubrics: define relevant issues, rules, standards, analytical steps, exceptions, risks, and critical errors.
• Evaluate AI-generated analyses for accuracy, completeness, reasoning quality, relevance, and professional rigor.
• Review issue spotting: determine whether the model identifies material legal or regulatory considerations.
• Assess reasoning quality: identify unsupported conclusions, incorrect rule application, missing exceptions, and faulty assumptions.
• Critique peer rubrics: review rubrics for gaps, ambiguity, redundancy, or inaccuracies in expectations.
• Develop reference content: create high-quality analyses, answer frameworks, critiques, and gold-standard examples.
• Support AI training: produce questions, rubrics, evaluations, preference data, and reference-answer pairs for RL and SFT workflows.
• Maintain professional quality: ensure content distinguishes factual analysis, legal interpretation, judgment, and uncertainty.

Conocimientos

Legal analysis

Regulatory interpretation

Issue spotting

Rubrics design

Professional judgment

Risk assessment

Policy interpretation

Contracts

Privacy/compliance

Educación

JD / LLB / LLM or equivalent

Descripción del empleo

SME Careers, a subsidiary of SuperAnnotate, is seeking a Legal & Compliance SME for an hourly remote contractor role. You will create realistic scenarios, evaluate AI-generated content, and design rigorous rubrics for professional analysis.

You will translate nuanced legal judgment into structured criteria covering issues, standards, risk, jurisdiction, and reasoning. This is a remote, international engagement with emphasis on accuracy and professional rigor in AI training data workflows.