Insurance AI Evaluation Specialist

Obsidian; San Francisco (CA); Remote; USD 120,000 - 180,000; Full time

Job summary

Cincinnatus LLC is seeking an Insurance SME to join a leading AI lab's GenAI team in San Francisco. You will evaluate AI outputs against rubrics and contribute real-world underwriting judgment to training data for foundational AI models.

The role requires 8+ years in insurance, strong communication, and reliable 35-hour weekday availability as part of a W-2 engagement with potential placement at a premier AI lab.

Qualifications

• 8+ years of dedicated professional experience in insurance (underwriting, claims, actuarial, risk management) at a top-tier organization.
• Hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria.
• Demonstrated career progression (e.g., Underwriter → Senior Underwriter → VP of Underwriting).
• Ability to engage reliably for at least 35 hours/week during weekdays.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in underwriting, claims, and risk-assessment reasoning.
• Design challenging, domain-relevant insurance tasks and write accurate, well-reasoned solutions grounded in real underwriting/claims practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to insurance tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Skills

Insurance expertise

LLM evaluation experience

Communication skills

Availability 35h/wk

Job description

Cincinnatus LLC is seeking an Insurance SME to join a leading AI lab's GenAI team in San Francisco. You will evaluate AI outputs against rubrics and contribute real-world underwriting judgment to training data for foundational AI models.

The role requires 8+ years in insurance, strong communication, and reliable 35-hour weekday availability as part of a W-2 engagement with potential placement at a premier AI lab.