Director, AI Evaluation & Quality

Figma; United States; On-site; USD 258,000 - 348,000; Full time

Job summary

Figma is looking for a Director of Research – AI Evals to define what “good” means for our AI‑powered experiences and to build the evaluation framework that informs what we ship. You’ll lead AI evaluation programs, blending human and automated approaches, and collaborate across Product, Design, Engineering, and Data Science to ensure trusted signals guide feature development.

You will sit upstream of many AI shipping decisions at Figma, influencing quality standards and partnering with teams to

Qualifications

• 10+ years of experience in product, research, applied research, or a closely related field, including 2+ years of management experience.
• Direct, hands-on experience owning the evaluation of AI/LLM-powered products.
• Expertise designing and running AI evaluation—human evaluation programs, rubric and benchmark/golden-dataset construction, inter-rater reliability.
• Strong data-driven mindset and comfort with metrics to reason about model behavior.
• Proven ability to influence senior stakeholders and drive evaluation as a strategic product function.

Responsibilities

• Own AI evaluation methods and operations for Figma's AI-powered experiences—define quality dimensions, design how we measure them, and turn results into decision-ready signals.
• Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars, combining human evaluation with automated/model-based approaches.
• Partner with engineering to stand up repeatable evaluation pipelines and regression testing, so evaluation is a routine part of how AI features are built and shipped.
• Produce clear readouts and dashboards that let stakeholders confidently make go/no-go and prioritization decisions.
• Socialize a shared definition of quality so evaluation standards are adopted across teams and advocate for evaluation as a strategic product partner.
• Manage a small team to execute our AI evals in partnership with contractors, internal staff, and/or LLMs.

Skills

AI evaluation

Leadership

Cross-functional collaboration

Qualitative & quantitative methods

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma is looking for a Director of Research – AI Evals to define what “good” means for our AI‑powered experiences and to build the evaluation framework that informs what we ship. You’ll lead AI evaluation programs, blending human and automated approaches, and collaborate across Product, Design, Engineering, and Data Science to ensure trusted signals guide feature development.

You will sit upstream of many AI shipping decisions at Figma, influencing quality standards and partnering with teams to