Insurance AI Evaluation Specialist for GenAI Training

Mercor; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Cincinnatus LLC is seeking a senior Insurance SME to join a leading AI lab's GenAI team in San Francisco. You will guide underwriting-focused evaluation of AI model outputs, develop scoring rubrics, and work with cross-functional teams to ensure high-quality training data for advanced insurance tasks.

This W-2 position places you within client teams as part of their extended workforce, with a strong emphasis on rigor, real-world underwriting judgment, and reliable collaboration.

Qualifications

• 8+ years of dedicated professional experience in insurance (underwriting, claims, actuarial, or risk management).
• Hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria.
• Clear career progression (e.g., Underwriter to VP of Underwriting).

Responsibilities

• Guide research and engineering teams to close knowledge gaps in underwriting, claims, and risk-assessment reasoning.
• Design challenging, domain-relevant insurance tasks and write accurate, well-reasoned solutions grounded in real underwriting/claims practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to insurance tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Skills

Strong verbal & written communication

Problem-solving

Interpersonal skills

Insurance domain knowledge

8+ years Insurance experience

Job description

Cincinnatus LLC is seeking a senior Insurance SME to join a leading AI lab's GenAI team in San Francisco. You will guide underwriting-focused evaluation of AI model outputs, develop scoring rubrics, and work with cross-functional teams to ensure high-quality training data for advanced insurance tasks.

This W-2 position places you within client teams as part of their extended workforce, with a strong emphasis on rigor, real-world underwriting judgment, and reliable collaboration.