AI Quality & Evaluation Lead

Millwonder; United States; Remote; USD 90,000 - 130,000; Full time

Benefits offered by this job

Remote, global work setup

Job summary

Millwonder is seeking a Lead AI Evaluation role to design and drive evaluation methods across accuracy, safety, consistency, and practical use. You will combine hands-on evaluation work with analytical leadership and direct client communication.

You will build evaluation frameworks, translate stakeholder requirements into measurable rubrics, assess model performance across key metrics, and present findings to clients in a clear, actionable way.

Qualifications

• Experience in AI evaluation, quality assurance, or applied research.
• Ability to translate requirements into measurable criteria.
• Strong analytical judgment and leadership.
• Clear client-facing communication.

Responsibilities

• Build evaluation frameworks for assessing model quality and behavior.
• Turn stakeholder requirements into clear, measurable scoring rubrics.
• Assess model accuracy, safety, consistency, and real-world performance.
• Communicate findings and evaluation approaches to clients.

Skills

AI evaluation

Quality assurance

Applied research

Measurable criteria

Analytical leadership

Client-facing communication

Job description

Role overview

Lead the design of evaluation methods that show how well AI systems perform across accuracy, safety, consistency, and practical use. The role combines hands-on evaluation work with analytical leadership and communication with clients.

Responsibilities

• Build evaluation frameworks for assessing model quality and behavior.
• Turn stakeholder requirements into clear, measurable scoring rubrics.
• Assess model accuracy, safety, consistency, and real-world performance.
• Communicate findings and evaluation approaches to clients.

Requirements

• Experience in AI evaluation, quality assurance, or applied research.
• Ability to translate requirements into measurable criteria.
• Strong analytical judgment and leadership.
• Clear client-facing communication.

Benefits and work setup

• Full-time role with a remote, global work setup.