Senior AI Evaluation Researcher (Remote)

Figma; San Francisco, New York (CA, NY); Remote; USD 169,000 - 250,000; Full time

Benefits offered by this job

Equity

Health insurance

Dental and vision coverage

Retirement plan with company match

Parental leave and family planning

Mental health benefits

Paid time off

Job summary

Figma is seeking a Researcher, AI Evaluations within the AI Insights team. This senior IC role focuses on strengthening evaluation methods for AI models and features through hands-on studies, robust rubrics, and reliable metrics that reflect real user needs.

You will partner with AI Research and Engineering to drive product quality and responsible AI practices. The role can be performed from a US hub or remotely in the United States, reporting to the AI Insights lead.

Qualifications

• 7+ years of experience leading evaluation projects with measurable impact.
• Experience evaluating AI products and translating findings into model/product improvements.
• Expertise in designing human evaluations, rubrics, datasets, benchmarks, and measuring inter-rater agreement.

Responsibilities

• Own evaluation research for priority AI Research workstreams; define quality criteria, design studies, analyze results, and recommend actions.
• Build rubrics, evaluation datasets, benchmarks, and quality thresholds reflecting user needs and expert design judgment.
• Collaborate with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and tests.
• Explain quality, failure patterns, and uncertainty; recommend actions guiding model selection, improvements, and release decisions.
• Validate automated graders against human ratings; identify bias, gaps, and uncertainty.
• Share reusable methods and mentor peers across teams.

Skills

AI evaluation

Research leadership

Qualitative methods

Quantitative methods

Experiment design

Inter-rater reliability

Stakeholder management

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma is seeking a Researcher, AI Evaluations within the AI Insights team. This senior IC role focuses on strengthening evaluation methods for AI models and features through hands-on studies, robust rubrics, and reliable metrics that reflect real user needs.

You will partner with AI Research and Engineering to drive product quality and responsible AI practices. The role can be performed from a US hub or remotely in the United States, reporting to the AI Insights lead.