Attorney Annotators for Legal LLM Evaluation

OpenTrain AI; Canada; Hybrid; CAD 3,800 - 7,600; Part time

Benefits offered by this job

Payment per label: 2.125 USD

Remote work

Job summary

OpenTrain AI is hiring an Attorney LLM Training Evaluator. You will interact with an internal LLM, ask legal questions, and label its responses using an A/B comparison workflow. The role is entry-level with English fluency and a legal degree from an accredited university in an English-speaking nation.

OpenTrain offers remote, contractor-based, part-time engagement with pay per label. Location eligibility includes multiple English-speaking countries, including Canada.

Qualifications

• English fluency required for all project work.
• Legal degree from an accredited university is required.
• Degree earned in an English-speaking nation is required.

Responsibilities

• Ask the model legal questions and label its responses using an A/B comparison approach.
• Review responses to legal questions and documents.
• Perform A/B labeling of model responses.
• Support evaluation rating and fine-tuning activities.

Skills

English fluency

Education

Legal degree from an accredited university

Degree earned in an English-speaking nation

Job description

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the #1 platform for finding and building careers in AI training and data labeling, helping people discover projects, build their profiles, and apply in minutes.

Creating an OpenTrain account is free, and this role offers a way for qualified attorneys to contribute to the development of modern language models.

• Remote AI training and data-labeling opportunity
• Contractor and part-time engagement
• Apply through OpenTrain

About AI Training Work

AI training is the human side of building artificial intelligence. People with professional and academic expertise help models improve by creating examples, reviewing outputs, and evaluating how accurately systems respond to real-world questions.

In this project, your legal knowledge will help assess model responses to legal questions. This work contributes to the evaluation and fine-tuning of generative AI systems.

• Work directly with generative AI evaluation tasks
• Use legal expertise to assess model-generated responses
• Participate in a fast-growing, cutting-edge technology field

The Role

As an Attorney LLM Training Evaluator, you will work with the project's internal large language model. You will ask legal questions and then label its responses using an A/B comparison approach.

The role is listed as entry level and requires English fluency, along with a legal degree from an accredited university in an English-speaking nation.

• Category: Legal questions and documents
• Data type: Text
• Work types: Evaluation rating and fine-tuning
• Payment: $2.125 USD per label
• Current labelers average $2,000 USD per week, though this is not a guaranteed amount

What You'll Do

Your core task is to interact with the internal LLM through legal prompts and evaluate the quality of its answers. You will compare responses and select the stronger option according to the project's labeling workflow.

• Ask the model legal questions
• Review responses to legal questions and documents
• Perform A/B labeling of model responses
• Support evaluation rating and fine-tuning activitiesComplete less than 20 hours of work per week
• Work without a weekly project cap

Requirements

Applicants must hold a legal degree from an accredited university in an English-speaking nation. English fluency is required for all project work.

• Legal degree from an accredited university
• Degree earned in an English-speaking nation
• English fluency
• Location in Australia, Canada, Ireland, the United Kingdom, or the United States
• Entry-level applicants are eligible

Who Should Apply

This opportunity is designed for attorneys and other legally trained applicants who want to apply their subject-matter knowledge to generative AI evaluation. It may suit candidates seeking flexible, part-time contractor work in a rapidly growing area of technology.

• Attorneys with a qualifying legal degree
• Applicants comfortable evaluating written legal responses
• Candidates interested in AI training and language-model development
• People seeking a flexible commitment of less than 20 hours per week