Remote LLM Evaluator & Prompt Engineer (Contract)

Benture

San Francisco (CA)

Remote

USD 6,700 - 27,000

Part time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Turing is seeking detail-oriented LLM Annotators for a remote, short-term contract role. You will evaluate and improve cutting-edge language models, design prompts, and validate outputs against source data.

Ideal candidates hold a Master's degree and have strong analytical and communication skills. The position offers flexible hours (20, 30, or 40 per week) with overlap for collaboration, and no medical or paid leave benefits as a contractor role.

Qualifications

  • Master's degree or higher in any discipline.
  • Minimum 3 years of professional, research, or teaching experience.
  • Excellent written English communication skills.
  • Exceptional attention to detail and ability to validate information against source data.

Responsibilities

  • Design challenging prompts that evaluate an LLM's ability to retrieve, analyze, and reason over structured data.
  • Assess AI-generated responses for factual accuracy, logical reasoning, and completeness.
  • Identify model failures, hallucinations, inconsistencies, and reasoning gaps.
  • Validate model outputs using provided datasets and supporting evidence.
  • Document findings with clear, evidence-based explanations.
  • Adhere consistently to annotation guidelines and maintain high-quality standards.

Skills

Analytical thinking
Attention to detail
Written English

Education

Master's degree or higher

Tools

CSV
Excel
Databases

Job description

Turing is seeking detail-oriented LLM Annotators for a remote, short-term contract role. You will evaluate and improve cutting-edge language models, design prompts, and validate outputs against source data.

Ideal candidates hold a Master's degree and have strong analytical and communication skills. The position offers flexible hours (20, 30, or 40 per week) with overlap for collaboration, and no medical or paid leave benefits as a contractor role.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

LLM Annotator - Master's Degree · Turing Turing · Varies · remote · 2w ago Varies 2w ago
LLM Annotator - Master's Degree · Turing Turing · Varies · remote · 2w ago Varies 2w ago

Benture • San Francisco (CA)

Remote
USD 6,700 - 27,000
Vietnamese LLM Prompt & Evaluation Specialist (Remote)
Vietnamese LLM Prompt & Evaluation Specialist (Remote)

Benture • San Francisco (CA)

Remote
USD 25,000 - 60,000
Fully remote
Flexible hours
Competitive compensation
Remote Prompt Engineer — LLM Production & Evaluation
Remote Prompt Engineer — LLM Production & Evaluation

Syntas • United States

Remote
USD 130,000 - 190,000
Remote within US
Competitive salary
MacBook Pro and peripherals
+4
LLM Annotator – Vietnamese · Turing Turing · Varies · remote · 1w ago Varies 1w ago
LLM Annotator – Vietnamese · Turing Turing · Varies · remote · 1w ago Varies 1w ago

Benture • San Francisco (CA)

Remote
USD 25,000 - 60,000
Fully remote
Flexible hours
Competitive compensation
Remote Prompt Writer & LLM Evaluation Expert
Remote Prompt Writer & LLM Evaluation Expert

OpenTrain AI, Inc. • United States

Remote
USD 83,000 - 138,000
Flexible schedule
Remote-friendly
LLM Evaluation & Prompt Testing Engineer
LLM Evaluation & Prompt Testing Engineer

HumanitApp • San Francisco (CA)

On-site
USD 120,000 - 180,000
Bilingual AI Analyst (Chinese/English) Remote
Bilingual AI Analyst (Chinese/English) Remote

Turing • United States

Remote
USD 34,000 - 69,000
Remote work
Flexible hours
Contractor engagement
Remote Linguistics Evaluation Specialist (Contract)
Remote Linguistics Evaluation Specialist (Contract)

AfterQuery • United States

Remote
USD 69,000 - 138,000
Weekly pay via Stripe
Fully remote
Flexible hours
Remote AI Prompt Engineer – LLM Testing
Remote AI Prompt Engineer – LLM Testing

AI Work Expert • San Francisco (CA)

Hybrid
USD 55,000 - 83,000
Remote AI Research Analyst: Advanced Math & LLM Tuning
Remote AI Research Analyst: Advanced Math & LLM Tuning

Turing • United States

Remote
USD 28,000 - 50,000
Fully remote work
Cutting-edge AI projects
Contract extension potential