Remote AI Legal Evaluator & Task Designer

Planet Pharma; San Francisco (CA); Remote; USD 140,000 - 190,000; Full time

Job summary

Planet Pharma seeks experienced legal professionals to evaluate how frontier AI models handle real legal work, including memos, contract review, drafting, and compliance analyses. You will design tasks, run them through frontier AI agents, and assess outputs against professional standards in a fully remote setting.

You will work with realistic practitioner files, building scenarios that test the AI’s ability to meet legal standards across litigation, corporate, compliance, and IP work.

Qualifications

• 2+ years legal experience preferred.
• Depth in litigation, corporate/transactional, compliance/regulatory, IP, employment, real estate or tax.
• Ground conclusions in primary sources: statutes, regulations, case law, and the contract in front of you.
• Bar admission valued but not required; JD/LLB preferred.
• US, UK, Canadian, Australian and New Zealand trained lawyers convert well.

Responsibilities

• Design challenging, realistic legal tasks drawn from your day-to-day work.
• Run tasks through frontier AI models and evaluate deliverables against professional standards.
• Compare two model outputs and document which performed better.
• Write detailed grading rubrics and explain why a response passes or fails.
• Flag concrete failures with evidence of misread authorities or missed issues.
• Contribute across litigation, transactional, compliance and regulatory, and IP work.

Skills

Legal analysis

Drafting

Regulatory compliance

Attention to detail

English writing

Education

JD or equivalent

Bachelor’s degree

Bar admission valued

Tools

Google Workspace

Claude

ChatGPT

LLM tools

Job description

Planet Pharma seeks experienced legal professionals to evaluate how frontier AI models handle real legal work, including memos, contract review, drafting, and compliance analyses. You will design tasks, run them through frontier AI agents, and assess outputs against professional standards in a fully remote setting.

You will work with realistic practitioner files, building scenarios that test the AI’s ability to meet legal standards across litigation, corporate, compliance, and IP work.