Regulatory Compliance Benchmark Designer (Remote)

Synthires; United States; Remote; USD 102,000 - 146,000; Part time

Job summary

Synthires is seeking experienced legal and compliance practitioners to design realistic, multi-step benchmark tasks for evaluating advanced AI systems. You will transform real-world workflows into challenging tasks that require professional judgment across contract review, due diligence, regulatory compliance, and risk analysis.

Remote, project-based contractor role with compensation up to $90 per hour. Candidates should have at least five years of hands-on experience and strong English

Qualifications

• Minimum five years hands-on experience in contract review, due diligence, regulatory compliance, or risk analysis.
• Experience interpreting legal documents and supporting materials.
• Strong analytical reasoning and the ability to identify inconsistencies across documents.
• Excellent written and spoken English (B2+; C1 preferred).

Responsibilities

• Design multi-step tasks based on real legal and compliance workflows.
• Develop task instructions and provide reference materials such as agreements and regulatory texts.
• Define objective grading criteria for consistent evaluation.
• Create tasks that assess legal judgment and document interpretation.
• Identify inconsistencies and substantive issues across multiple documents.
• Ensure tasks are clearly defined, challenging, and gradable.
• Flag ambiguities or ungradable tasks and avoid submission for approval.
• Deliver approved tasks on schedule with quality standards.

Skills

Analytical reasoning

Independent work

English proficiency

Education

Law degree or equivalent compliance qualification

Tools

Legal document interpretation

Job description

Synthires is seeking experienced legal and compliance practitioners to design realistic, multi-step benchmark tasks for evaluating advanced AI systems. You will transform real-world workflows into challenging tasks that require professional judgment across contract review, due diligence, regulatory compliance, and risk analysis.

Remote, project-based contractor role with compensation up to $90 per hour. Candidates should have at least five years of hands-on experience and strong English