Head of AI Evaluation & Insurance Quality Operations

Jobgether; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Jobgether seeks an operational leader to head its AI evaluation function, overseeing team development, quality governance, workflow performance, and continuous improvement. You will design an operational framework, scale a domain-focused evaluator team, and align practices with client SLAs and policy standards.

You will partner with product and ML engineering to translate insurance insights into actionable prompts, calibration, and evaluation improvements, ensuring high-quality outputs and

Responsibilities

• Lead program stand-up and scale the AI evaluation function to meet client demand.
• Design and implement the operational framework for AI evaluation.
• Launch and manage an initial team of about 10 insurance domain evaluators.
• Scale team capacity and infrastructure in response to client demand.
• Monitor launch speed, queue capacity, and reviewer time-to-proficiency.
• Establish productivity standards, quality audit rubrics, and calibration procedures.
• Conduct secondary reviews of complex insurance cases and coach improvement.
• Audit policy edge cases and identify recurring errors in outputs.
• Track quality metrics and reduce error rates across active queues.
• Build and manage daily workflow queues to distribute evaluation tasks.
• Monitor reviewer throughput, productivity, and capacity.
• Optimize resource allocation to meet client turnaround times and SLAs.
• Identify and eliminate workflow bottlenecks while preserving quality.
• Consolidate audit findings and identify systemic AI failure patterns.
• Lead feedback sessions with product and ML engineering teams.
• Translate insurance domain insights into actionable evaluation improvements.
• Track defect reporting quality and model accuracy improvements.
• Recruit, mentor, and evaluate junior and senior evaluators.
• Partner with learning teams to create ongoing training programs.
• Ensure evaluators stay updated on policy standards and AI tools.
• Monitor team retention, skills development, performance, and calibration consistency.

Job description

Jobgether seeks an operational leader to head its AI evaluation function, overseeing team development, quality governance, workflow performance, and continuous improvement. You will design an operational framework, scale a domain-focused evaluator team, and align practices with client SLAs and policy standards.

You will partner with product and ML engineering to translate insurance insights into actionable prompts, calibration, and evaluation improvements, ensuring high-quality outputs and