LLM Evaluation Domain Expert · Turing Turing · Varies · remote · 4d ago Varies 4d ago

Benture; San Francisco (CA); A distancia; USD 90.000 - 130.000; Jornada completa

Descripción de la vacante

Turing, based in San Francisco, is recruiting Domain Experts to support evaluation and improvement of advanced LLMs. You will craft complex prompts, test outputs for accuracy and reasoning, and identify gaps to help advance AI research and deployment.

Ideal candidates have strong writing and analytical skills, a PhD-level mindset, and experience in AI evaluation. This role offers remote work with flexible scheduling and diverse, impactful projects.

Formación

• Master's degree or higher in a relevant field is required.
• At least three years of professional, research or teaching experience.
• Excellent written English communication skills.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.

Responsabilidades

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance and overall quality.
• Identify factual errors, logical inconsistencies and knowledge gaps in outputs.
• Develop challenging test cases and edge cases to expose model limitations.
• Provide clear, detailed, evidence-based feedback to improve model performance.

Conocimientos

Written English

Analytical thinking

Critical thinking

Attention to detail

Educación

Master's degree or higher in a relevant field

Descripción del empleo

Domain Expert, LLM Evaluation | Pay: Varies | Remote

About the Role

Turing, based in San Francisco, works with frontier AI labs and enterprises on AI research and deployment. The company is hiring Domain Experts (SMEs) to support the evaluation, testing and improvement of advanced Large Language Models (LLMs). You will apply your subject-matter expertise to assess AI-generated responses, identify knowledge gaps, design challenging prompts, and improve model accuracy, reliability and reasoning.

What the Work Involves

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance and overall quality.
• Identify factual errors, logical inconsistencies, knowledge gaps and model weaknesses.
• Develop challenging test cases and edge cases that expose model limitations.
• Provide clear, detailed, evidence-based feedback to improve model performance.

Minimum Qualifications

• Master's degree or higher in a relevant field.
• At least three years of professional, research or teaching experience.
• Excellent written English communication skills.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.

Preferred Qualifications

• Experience working with LLMs or Generative AI.
• Familiarity with prompt engineering, AI evaluation or model testing.
• Experience designing challenging prompts, identifying edge cases or stress-testing AI models.

Ideal Candidate

Curious, detail-oriented and deeply knowledgeable in your field. You enjoy solving complex problems, challenging assumptions, and identifying subtle factual or reasoning errors.

Pay, Schedule and Location

The listing does not specify pay, working hours, commitment or location restrictions.