Senior AI Evaluation Researcher (Remote)

Figma; United States; Remote; USD 169,000 - 250,000; Full time

Benefits offered by this job

Equity

Health insurance

Parental leave

Paid time off

Annual bonus

Job summary

Figma is seeking a Senior Researcher, AI Evaluations to join AI Insights within the Research team. This senior IC role strengthens the team’s evaluation through hands-on studies, rubrics, and reliable measures of quality reflecting user needs, guiding model and product improvements.

You will own evaluation research for priority AI workstreams, build datasets and benchmarks, and work with AI Research and Modeling Platform engineers to integrate evaluated methods into repeatable evaluations and

Qualifications

• 7+ years of experience in research or a related field with independent leadership of evaluation projects.
• Hands-on experience evaluating AI products and translating findings into model/product improvements.
• Expertise in designing human evaluations, scoring rubrics, datasets, and benchmarks with inter-rater reliability.
• Strong qualitative and quantitative research skills to analyze data, interpret uncertainty and derive insights.
• Excellent problem-solving and stakeholder management to influence cross-functional decisions.

Responsibilities

• Own evaluation research for priority AI Workstreams; define quality criteria, design studies, analyze results and recommend actions.
• Build rubrics, evaluation datasets, benchmarks and quality thresholds reflecting user needs and expert design judgment.
• Collaborate with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and tests.
• Explain quality, failure patterns and uncertainty; guide model selection, improvements, and release decisions.
• Validate automated graders against human ratings; identify bias, gaps and uncertainty.
• Connect user research with AI Research’s model development; share reusable methods and mentor peers.

Skills

Evaluation research

Rubrics design

Benchmarks

Human evaluation design

Statistical analysis

Stakeholder management

Experimentation

AI product evaluation

Education

Advanced training in quantitative/behavioral discipline

Formal education or equivalent hands-on experience

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma is seeking a Senior Researcher, AI Evaluations to join AI Insights within the Research team. This senior IC role strengthens the team’s evaluation through hands-on studies, rubrics, and reliable measures of quality reflecting user needs, guiding model and product improvements.

You will own evaluation research for priority AI workstreams, build datasets and benchmarks, and work with AI Research and Modeling Platform engineers to integrate evaluated methods into repeatable evaluations and