Remote LLM Evaluation Domain Expert

Benture

San Francisco (CA)

Hybrid

USD 90,000 - 150,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Turing, based in San Francisco, is hiring Domain Experts (SMEs) to support evaluation, testing and improvement of advanced Large Language Models (LLMs). You will use your subject-matter expertise to assess AI-generated responses, design challenging prompts and provide evidence-based feedback to enhance model accuracy and reliability.

Ideal candidates hold a Master’s or higher, have at least three years of professional or academic experience, and excel in precise written English and analytical

Qualifications

  • Master's degree or higher in a relevant field.
  • At least three years of professional, research or teaching experience.
  • Excellent written English communication skills.

Responsibilities

  • Create complex, domain-specific prompts to test LLM capabilities.
  • Evaluate AI-generated responses for factual accuracy, reasoning, relevance and overall quality.
  • Identify factual errors, logical inconsistencies, knowledge gaps and model weaknesses.
  • Develop challenging test cases and edge cases that expose model limitations.
  • Provide clear, detailed, evidence-based feedback to improve model performance.

Skills

Excellent written English
Analytical thinking
Attention to detail

Education

Master's degree or higher in a relevant field

Job description

Turing, based in San Francisco, is hiring Domain Experts (SMEs) to support evaluation, testing and improvement of advanced Large Language Models (LLMs). You will use your subject-matter expertise to assess AI-generated responses, design challenging prompts and provide evidence-based feedback to enhance model accuracy and reliability.

Ideal candidates hold a Master’s or higher, have at least three years of professional or academic experience, and excel in precise written English and analytical

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

LLM Evaluation Domain Expert · Turing Turing · Varies · remote · 4w ago Varies 4w ago
LLM Evaluation Domain Expert · Turing Turing · Varies · remote · 4w ago Varies 4w ago

Benture • San Francisco (CA)

Hybrid
USD 90,000 - 150,000
LLM Prompting and Evaluation Lead
LLM Prompting and Evaluation Lead

Turing • United States

Remote
USD 120,000 - 150,000
LLM Evaluation & Prompt Testing Engineer
LLM Evaluation & Prompt Testing Engineer

HumanitApp • San Francisco (CA)

On-site
USD 120,000 - 180,000
Tooling Engineer
Tooling Engineer

Turing • United States

Remote
USD 120,000 - 150,000
Remote AI Trainer & Evaluator for LLMs (RLHF)
Remote AI Trainer & Evaluator for LLMs (RLHF)

prolificacademicltd • United States

Remote
USD 91,000 - 129,000
Remote work
Flexible hours
Up to $80/hour
ML Engineer - LLM & Agentic Evaluations
ML Engineer - LLM & Agentic Evaluations

Turing • United States

Remote
USD 34,000 - 55,000
Fully remote work
AI Engineer - LLM Training & Evaluation (Remote)
AI Engineer - LLM Training & Evaluation (Remote)

Prolific • Memphis (TN)

Hybrid
USD 90,000 - 130,000
Competitive pay rates
Flexible hours
Ability to work from home
Remote Physics Expert for LLM Evaluation
Remote Physics Expert for LLM Evaluation

Turing • United States

Remote
USD 83,000 - 165,000
Work in a fully remote environment
Opportunity to work on cutting-edge AI
Contract extension based on project
Domain Expert - Art
Domain Expert - Art

Turing • United States

Remote
USD 55,000 - 110,000
Contractor assignment
8 weeks duration
Remote Senior AI/ML Engineer – LLM Training & Evaluation
Remote Senior AI/ML Engineer – LLM Training & Evaluation

Prolific • Milwaukee (WI)

Remote
USD 100,000 - 130,000
Competitive pay rates
Flexible hours
Ability to work from home