LLM Prompting and Evaluation Lead

Turing; United States; Remote; USD 120,000 - 150,000; Full time

Job summary

Turing in San Francisco, CA seeks Domain Experts (SMEs) to support evaluation, testing, and improvement of advanced LLMs. You will apply your expertise to assess AI responses, design challenging prompts, and improve model accuracy and reliability.

Ideal candidates have a Master’s degree or higher, 3+ years of professional or research experience, and excellent written English. Strong analytical thinking and meticulousness are essential, along with experience in prompt engineering or AI evaluation.

Qualifications

• Master's degree or higher in a relevant field.
• At least three years of professional, research, or teaching experience.
• Excellent written English communication skills.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.

Responsibilities

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Identify factual errors, logical inconsistencies, knowledge gaps, and model weaknesses.
• Develop challenging test cases and edge cases that expose model limitations.
• Provide clear, detailed, and evidence-based feedback to improve model performance.

Skills

Business Analysis

Critical Thinking

Attention to Detail

Education

Master's degree or higher in a relevant field

Job description

Turing in San Francisco, CA seeks Domain Experts (SMEs) to support evaluation, testing, and improvement of advanced LLMs. You will apply your expertise to assess AI responses, design challenging prompts, and improve model accuracy and reliability.

Ideal candidates have a Master’s degree or higher, 3+ years of professional or research experience, and excellent written English. Strong analytical thinking and meticulousness are essential, along with experience in prompt engineering or AI evaluation.