Sr. Legal AI Prompter, Evaluator

Thomson Reuters; Eagan, Frisco, New York (MN, TX, NY); Hybrid; USD 120,000 - 222,000; Full time

Benefits offered by this job

Hybrid work model

Career development and growth

Mental health days off

Job summary

Thomson Reuters seeks a Sr. Legal AI Prompter / Evaluator to advance CoCounsel Legal through prompts, skills, and agent configurations. You will experiment with Thomson and other LLMs, build evaluation schemes, and ensure outputs are accurate and legally sound.

You will partner with SMEs, software engineers, and data scientists to improve product quality. The ideal candidate is an experienced lawyer with a passion for technology, comfortable with AI risk assessment, and capable of translating

Qualifications

• Juris Doctor degree and bar admission in a US or Canadian jurisdiction.
• 5-7 years of combined experience, including 2-3 years legal practice and collaboration with AI/tech teams.
• Proven experience with generative AI, prompt engineering, and evaluating model outputs, plus coding in Python.

Responsibilities

• Create, test, and refine prompts, skills, and agent configurations for CoCounsel Legal.
• Experiment with Thomson and other LLMs to compare results and guide model selection.
• Build datasets with SMEs and design evaluation schemes with rubrics and metrics.
• Write and run scripts, experiments, and automated tests ensuring quality over large datasets.
• Monitor production performance and identify failure modes in prompting, tooling, or retrieval.
• Collaborate with engineers and data scientists to integrate LLMs into multi-step systems.
• Engage with legal professionals to gather insights and stay current on legal tech developments.
• Mentor colleagues and draft best-practice guidance for using LLMs.

Skills

Juris Doctor

Bar admission (US/Canada)

5-7 years total exp

Python & Git

Legal/AI experience

Education

Juris Doctor degree

Tools

OpenAI API

Anthropic API

Gemini API

LLM APIs

Job description

## Sr. Legal AI Prompter, EvaluatorApply: Hybrid: United States of America, Eagan, Minnesota: United States of America, Frisco, Texas: United States of America, New York, New York: Canada, Toronto, Ontario: Full time: Posted Today: End Date: December 31, 2026 (30+ days left to apply): JREQ205051**Sr. Legal AI Prompter / Evaluator** **Job Description** Sr. Legal AI Prompters / Evaluators are responsible for improving CoCounsel Legal by writing and refining prompts, skills, and agent configurations; experimenting with large language models (LLMs), including Thomson, our proprietary LLM; and evaluating the results. They partner closely with specialized legal subject matter experts (SMEs), software engineers, and data scientists to make sure our products deliver accurate, legally sound outputs that legal professionals can trust. As senior members of the team, they take a leading role in the hands-on development of LLM-driven solutions, design and run experiments to find out what works, and help raise the team’s capabilities. The ideal candidate is an experienced, practiced lawyer with a passion for technology and a strong understanding of emerging technologies in the legal profession, who can think critically about AI risks and opportunities and translate legal expertise into practical, high-quality AI outputs. **Key Responsibilities*** **Create**, test, and refine prompts, skills, and agent configurations for CoCounsel Legal to ensure accurate, legally sound, and contextually appropriate responses* **Experiment** with Thomson and other LLMs on legal tasks, comparing results to inform which model best fits each use case* **Build** gold-standard datasets with SMEs, coordinate evaluations, and design evaluation schemes, including rubrics, metrics, success criteria, and LLM-as-judge prompts, to test model outputs and inform launch readiness decisions* **Write** and **run** scripts, experiments, and automated tests that ensure quality and consistency over large datasets and time periods* **Monitor** production performance, analyze failure modes and identify where in the pipeline they originate (prompting, tooling, retrieval, or content), and provide strategic feedback to software engineers and data scientists* **Collaborate** with software engineers, data scientists, and cross-functional teams as they integrate LLMs into multi-step systems, and evaluate the feasibility and potential impact of new product ideas* **Engage** with legal professionals, industry leaders, and internal stakeholders to gather insights and validate product concepts, and stay informed about developments in legal technology and LLMs* **Mentor** colleagues and draft guidance materials, reusable prompts, and best-practice documentation on getting the most out of LLMs **Required Qualifications*** Juris Doctor degree or equivalent law degree and bar admission in a US or Canadian jurisdiction* 5-7 years of combined experience, including at least 2-3 years of legal practice, with the remaining years in related work such as collaborating with software engineers, building or testing products, applying generative AI, or writing code* Proven expertise applying generative AI, including prompt engineering and evaluating model outputs, and an understanding of how LLMs work and where they fail* Proficiency working in a Python code base and using Git, including writing, modifying, and debugging Python scripts, calling LLMs through code (e.g., OpenAI, Anthropic, Gemini), and using coding agents to build and iterate on that work* Excellent communication and collaboration skills, with the ability to work effectively with legal SMEs and across functions **Preferred Qualifications*** Have substantial experience working with and testing legal products and technology, including online legal research and practice tools* Have hands-on experience designing evaluations, such as rubrics, human grading frameworks, or LLM-as-judge prompts* Are familiar with agentic and RAG systems, including how retrieval, tool use, and prompting each affect output quality* Are a self-starter who is comfortable with ambiguity and learns new tools and models quickly #LI-DS4Replacement: This position is open due to an existing vacancy to support our evolving business needs. **What’s in it For You?*** **Hybrid Work Model:** We’ve adopted a flexible hybrid working environment for our office-based roles while delivering a seamless experience that is digitally and physically connected.* **Flexibility & Work-Life Balance:** Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance.* **Career Development and Growth:** By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrow’s challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future.* **Industry Competitive Benefits:** We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing.* **Culture:** Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our values: Obsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together.* **Social Impact:** Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives.* **Making a Real-World Impact:** We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world. Our use of AI within the recruitment process Thomson Reuters utilizes Artificial Intelligence (AI) to support parts of our global recruitment process. Unless you opt-out, our AI system will assess the information provided by you and compare it to the requirements listed for the role, and present the result to our recruitment personnel for further review. The AI system acts as a supporting tool, but there is always a human making the decision if you will be considered for the role.In the United States, Thomson Reuters offers a comprehensive benefits package to our employees. Our benefit package includes market competitive health, dental, vision, disability, and life insurance programs, as well as a competitive 401k plan with company match. In addition, Thomson Reuters offers market leading work life benefits with competitive vacation, sick and safe paid time off, paid holidays (including two company mental health days off), parental leave, sabbatical leave. These benefits meet or exceeds the requirements of paid time off in accordance with any applicable state or municipal laws. Finally, Thomson Reuters offers the following additional benefits: optional hospital, accident and sickness insurance paid 100% by the employee; optional life and AD&D insurance paid 100% by the employee; Flexible Spending and Health Savings Accounts; fitness reimbursement; access to Employee Assistance Program; Group Legal Identity Theft Protection benefit paid 100% by employee; access to 529 Plan; commuter benefits; Adoption & Surrogacy Assistance; Tuition Reimbursement; and access to Employee Stock Purchase Plan.Thomson Reuters complies with local laws that require upfront disclosure of the expected pay range for a position. The base compensation range varies across locations.Eligible office location(s) for this role include one or more of the following: New York City, San Francisco, Los Angeles, and/or Irvine, CA; McLean, VA; Washington, DC. The base compensation range for the role in any of those locations is $137,900 USD - $256,100 USD.For any eligible US locations, unless otherwise noted, the base compensation range for this role is $119,700 USD - $222,300 USD.For Ontario, Canada, the base compensation range for this role is $126,000 CAD - $176,000 CAD.Base pay is positioned within the range based on several factors including an individual’s knowledge, skills and experience with consideration given to internal equity. Base pay is one part of a comprehensive Total Reward program which also includes flexible and supportive benefits and other wellbeing programs.This role may also be eligible for an Annual Bonus based on a combination of enterprise and individual performance.