Director of AI Evaluation & Quality

Figma; San Francisco (CA); On-site; USD 258,000 - 348,000; Full time

Job summary

Figma's Research team is hiring a Director, Research – AI Evals to own how we measure the quality of AI-powered experiences. You’ll define what “good” means for our AI features, build the frameworks and quality bars to measure it, and turn that into trusted signals that product teams rely on to decide what to ship.

The ideal candidate brings deep hands-on experience evaluating AI/LLM-powered products—blending human evaluation with automated, model-based approaches—and can partner with product,

Qualifications

• 10+ years in product, research, or related field; 2+ years in management.
• Hands-on AI/LLM evaluation experience with human and automated methods.
• Strong qualitative and quantitative skills with data-driven mindset.
• Proven ability to align stakeholders and executive teams around quality standards.

Responsibilities

• Own AI evaluation methods and define quality dimensions for features.
• Build evaluation frameworks, rubrics, datasets, and quality bars.
• Partner with engineering to run repeatable evaluation pipelines.
• Create dashboards for go/no-go decisions and prioritization.
• Socialize quality standards across teams and advocate for evaluation.
• Manage a small team with contractors and internal staff.

Skills

10+ years exp in product, research, or

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma's Research team is hiring a Director, Research – AI Evals to own how we measure the quality of AI-powered experiences. You’ll define what “good” means for our AI features, build the frameworks and quality bars to measure it, and turn that into trusted signals that product teams rely on to decide what to ship.

The ideal candidate brings deep hands-on experience evaluating AI/LLM-powered products—blending human evaluation with automated, model-based approaches—and can partner with product,