Tooling Engineer

HumanitApp; San Francisco (CA); Sur place; USD 120 000 - 180 000; Plein temps

Résumé du poste

Turing, based in San Francisco, seeks Domain Experts to support evaluation and testing of advanced LLMs. You will craft domain-specific prompts, assess responses for accuracy and reasoning, and design edge cases to reveal model weaknesses.

This is a highly technical, detail-oriented role requiring strong analytical skills and experience with AI evaluation. The ideal candidate will have a Master’s degree or higher and 3+ years in research, academia, or industry, with familiarity in prompt

Qualifications

• Master’s degree or higher in a relevant field.
• At least three years of professional, research, or teaching experience.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.
• Experience with LLMs or Generative AI preferred.
• Familiarity with prompt engineering, AI evaluation, or model testing.

Responsabilités

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Identify factual errors, logical inconsistencies, knowledge gaps, and model weaknesses.
• Develop challenging test cases and edge cases that expose model limitations.
• Provide clear, detailed, and evidence-based feedback to improve model performance.

Connaissances

Domain expertise

LLMs / Generative AI

Prompt engineering

AI evaluation

Stress testing

Critical thinking

Attention to detail

Formation

Master's degree or higher in a relevant field

Description du poste

Future projects, not immediate vacancies. Check each network's country and experience requirements.

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.

About the Role

We are seeking highly qualified Domain Experts (SMEs) to support the evaluation, testing, and improvement of advanced Large Language Models (LLMs). You will apply your subject-matter expertise to assess AI-generated responses, identify knowledge gaps, design challenging prompts, and improve model accuracy, reliability, and reasoning.

Key Responsibilities

Create complex, domain-specific prompts to test LLM capabilities.

Evaluate AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.

Identify factual errors, logical inconsistencies, knowledge gaps, and model weaknesses.

Develop challenging test cases and edge cases that expose model limitations.

Provide clear, detailed, and evidence-based feedback to improve model performance.

Minimum Qualifications

Master’s degree or higher in a relevant field.

At least three years of professional, research, or teaching experience.

Strong analytical and critical-thinking abilities.

Exceptional attention to detail.

Preferred Qualifications

Experience working with LLMs or Generative AI.

Familiarity with prompt engineering, AI evaluation, or model testing.

Experience designing challenging prompts, identifying edge cases, or stress-testing AI models.

Ideal Candidate

The ideal candidate is curious, detail-oriented, and deeply knowledgeable in their field. They enjoy solving complex problems, challenging assumptions, and identifying subtle factual or reasoning errors. Experience designing prompts that uncover model limitations and improve AI performance will be a strong advantage.

Business Analysis

Compensation context

Compensation is not stated in the current feed. Confirm the rate and workload before accepting an assessment or project.

• Review the official description below for experience and equipment requirements.
• Confirm location eligibility, language requirements and weekly availability with the partner.

Not the right fit?

• Review the official description below for experience and equipment requirements.
• Confirm location eligibility, language requirements and weekly availability with the partner.

Compensation is not stated in the current feed. Confirm the rate and workload before accepting an assessment or project.

How should I compare the listed pay?

Compensation is not stated in the current feed. Confirm the rate and workload before accepting an assessment or project.

Does Apply use a referral link?

No. HumanitApp independently curates the opportunity. The partner platform manages applications and hiring decisions.

Is HumanitApp the employer?

No. HumanitApp independently curates the opportunity. The partner platform manages applications and hiring decisions.