Remote AI Compliance Benchmark Architect

Synthires; Greater London; Remote; GBP 77,000 - 109,000; Part time

Benefits offered by this job

Fully remote

Job summary

Synthires seeks experienced legal and compliance practitioners to design multi-step benchmark tasks for evaluating advanced AI systems. You will transform real-world contract review, due diligence, and regulatory workflows into clearly defined tasks, using your professional judgment to create objective grading criteria.

Experience interpreting legal documents, risk analysis, and strong English skills are essential.

Qualifications

• Minimum five years hands-on experience in contract review, due diligence or compliance.
• Ability to interpret legal documents and regulatory requirements.
• Strong written and spoken English, professional judgement.

Responsibilities

• Design multi-step tasks based on real legal and compliance workflows.
• Develop task instructions with reference materials.
• Define objective grading criteria for consistent evaluation.
• Create tasks that assess legal judgment and document interpretation.
• Identify inconsistencies and substantive issues across documents.
• Ensure tasks are clearly defined, challenging, and gradable.
• Flag ambiguous tasks and deliver on schedule with quality.

Skills

Contract review

Due diligence

Regulatory compliance

Risk analysis

Analytical reasoning

English proficiency

Education

Law degree

Bar admission

Job description

Synthires seeks experienced legal and compliance practitioners to design multi-step benchmark tasks for evaluating advanced AI systems. You will transform real-world contract review, due diligence, and regulatory workflows into clearly defined tasks, using your professional judgment to create objective grading criteria.

Experience interpreting legal documents, risk analysis, and strong English skills are essential.