Senior Legal AI Prompter & Evaluator (Hybrid)

Thomson Reuters

New York (NY)

Hybrid

USD 120,000 - 222,000

Full time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Hybrid work model
Mental health days
Tuition reimbursement
Employee incentive programs
Wellbeing resources

Job summary

Thomson Reuters is seeking a Sr. Legal AI Prompter/Evaluator in New York to lead hands‑on development of LLM‑driven legal tools.

You will design prompts, run experiments, and build evaluation schemes with SMEs, software engineers, and data scientists to ensure accurate and trustworthy outputs for legal professionals. The role requires a JD and 5–7 years of combined experience, including 2–3 years of legal practice, plus strong Python and Git skills.

Qualifications

  • Juris Doctor degree or equivalent law degree and bar admission in a US or Canadian jurisdiction.
  • 5–7 years of combined experience, including 2–3 years of legal practice.
  • Proven expertise applying generative AI, including prompt engineering and evaluating model outputs.
  • Proficiency working in a Python code base and using Git; ability to call LLMs through code (e.g., OpenAI, Anthropic, Gemini).
  • Excellent communication and collaboration skills, with the ability to work effectively with legal SMEs and across functions.

Responsibilities

  • Create, test, and refine prompts, skills, and agent configurations for CoCounsel Legal to ensure accurate, legally sound, and contextually appropriate responses.
  • Experiment with Thomson and other LLMs on legal tasks, comparing results to inform which model best fits each use case.
  • Build gold‑standard datasets with SMEs, coordinate evaluations, and design evaluation schemes, including rubrics, metrics, success criteria, and LLM‑as‑judge prompts, to test model outputs and inform launch readiness decisions.
  • Write and run scripts, experiments, and automated tests that ensure quality and consistency over large datasets and time periods.
  • Monitor production performance, analyze failure modes and identify where in the pipeline they originate (prompting, tooling, retrieval, or content), and provide strategic feedback to software engineers and data scientists.
  • Collaborate with software engineers, data scientists, and cross‑functional teams as they integrate LLMs into multi‑step systems, and evaluate the feasibility and potential impact of new product ideas.
  • Engage with legal professionals, industry leaders, and internal stakeholders to gather insights and validate product concepts, and stay informed about developments in legal technology and LLMs.
  • Mentor colleagues and draft guidance materials, reusable prompts, and best‑practice documentation on getting the most out of LLMs.

Skills

Prompt engineering
LLM evaluation
Python
Git
Communication

Education

Juris Doctor or equivalent

Tools

OpenAI API
Anthropic API
Gemini API

Job description

Thomson Reuters is seeking a Sr. Legal AI Prompter/Evaluator in New York to lead hands‑on development of LLM‑driven legal tools.

You will design prompts, run experiments, and build evaluation schemes with SMEs, software engineers, and data scientists to ensure accurate and trustworthy outputs for legal professionals. The role requires a JD and 5–7 years of combined experience, including 2–3 years of legal practice, plus strong Python and Git skills.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Legal AI Prompting & Evaluation Lead
Senior Legal AI Prompting & Evaluation Lead

Thomson Reuters • Frisco (TX)

Hybrid
USD 120,000 - 222,000
Hybrid work model
Career development opportunities
Mental health days
+3
Senior Legal AI Prompt Engineer & Evaluator
Senior Legal AI Prompt Engineer & Evaluator

Thomson Reuters • Eagan (MN)

Hybrid
USD 120,000 - 222,000
Hybrid Work Model
Flexible vacation
Mental Health Days
+3
Senior Legal AI Prompt Engineer & Evaluator
Senior Legal AI Prompt Engineer & Evaluator

USA018 Thomson Reuters Holdings Inc. • Eagan (MN)

Hybrid
USD 120,000 - 222,000
Hybrid Work Model
Flex My Way
Career Development
+2
Senior Legal AI Prompt Engineer and Evaluator
Senior Legal AI Prompt Engineer and Evaluator

Thomson Reuters  • Eagan (MN), Frisco (TX), New York (NY)

Hybrid
USD 120,000 - 222,000
Hybrid work model
Career development and growth
Mental health days off
Lead AI Solutions Specialist – Legal Tech (Hybrid)
Lead AI Solutions Specialist – Legal Tech (Hybrid)

TRSS • Washington

Hybrid
USD 120,000 - 222,000
Hybrid work model
Career development
Competitive benefits
+2
Lead AI Solutions Architect - Legal Research Tools
Lead AI Solutions Architect - Legal Research Tools

Thomson Reuters • Eagan (MN)

Hybrid
USD 120,000 - 222,000
Hybrid Work Model
Career Development and Growth
Comprehensive Benefits
+1
Senior Legal Engineer: AI-Powered Legal Solutions
Senior Legal Engineer: AI-Powered Legal Solutions

Thomson Reuters • New York (NY)

Hybrid
USD 118,000 - 219,000
Hybrid work model
Flex My Way policies
Two Mental Health Days off
+2
Senior AI Solutions Lead - Legal Tech (Hybrid)
Senior AI Solutions Lead - Legal Tech (Hybrid)

Thomson Reuters • Frisco (TX)

Hybrid
USD 120,000 - 222,000
Hybrid Work Model
Mental Health Days
Tuition Reimbursement
+1
Legal AI Engineer — Lead AI-Driven Legal Workflows
Legal AI Engineer — Lead AI-Driven Legal Workflows

Thomson Reuters  • New York (NY)

On-site
USD 100,000 - 186,000
Flex My Way
Grow My Way
Mental Health Days
+4
Hybrid Senior Legal AI Engineer - Cloud-Native & LLM
Hybrid Senior Legal AI Engineer - Cloud-Native & LLM

Thomson Reuters Foundation • Frisco (TX)

On-site
USD 102,000 - 190,000
Hybrid Work Model
Flexible work policies
Comprehensive benefits