LLM Evaluation & Prompt Testing Specialist (Remote)

Benture; San Francisco (CA); Remote; USD 90,000 - 130,000; Full time

Job summary

Turing, based in San Francisco, is recruiting Domain Experts to support evaluation and improvement of advanced LLMs. You will craft complex prompts, test outputs for accuracy and reasoning, and identify gaps to help advance AI research and deployment.

Ideal candidates have strong writing and analytical skills, a PhD-level mindset, and experience in AI evaluation. This role offers remote work with flexible scheduling and diverse, impactful projects.

Qualifications

• Master's degree or higher in a relevant field is required.
• At least three years of professional, research or teaching experience.
• Excellent written English communication skills.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.

Responsibilities

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance and overall quality.
• Identify factual errors, logical inconsistencies and knowledge gaps in outputs.
• Develop challenging test cases and edge cases to expose model limitations.
• Provide clear, detailed, evidence-based feedback to improve model performance.

Skills

Written English

Analytical thinking

Critical thinking

Attention to detail

Education

Master's degree or higher in a relevant field

Job description

Turing, based in San Francisco, is recruiting Domain Experts to support evaluation and improvement of advanced LLMs. You will craft complex prompts, test outputs for accuracy and reasoning, and identify gaps to help advance AI research and deployment.

Ideal candidates have strong writing and analytical skills, a PhD-level mindset, and experience in AI evaluation. This role offers remote work with flexible scheduling and diverse, impactful projects.