Senior AI Evaluation Researcher (Remote)

ContentBuffer; San Francisco (CA); Remote; USD 169,000 - 250,000; Full time

Benefits offered by this job

Equity

Health insurance

Dental coverage

Vision coverage

Retirement benefits

Parental leave

Paid time off (PTO)

Annual bonus

Job summary

Figma is seeking a senior Researcher, AI Evaluations to join the AI Insights team. This individual contributor role focuses on conducting evaluation studies, building rubrics, datasets, and benchmarks to guide model and product improvements.

You’ll work with AI Research and Modeling Platform engineers to ensure repeatable evaluations and high-quality results. You’ll apply strong qualitative and quantitative methods to measure alignment, uncertainty, and bias, influencing cross-functional

Qualifications

• 7+ years of experience leading evaluation projects with measurable impact.
• Hands-on experience evaluating AI products and translating findings into model/product improvements.
• Expertise in designing human evaluations, scoring rubrics, datasets, and benchmarks.

Responsibilities

• Own evaluation research for priority AI workstreams; define quality criteria and analyze results.
• Build rubrics, evaluation datasets, benchmarks, and quality thresholds reflecting user needs.
• Collaborate with AI Research and Modeling Platform engineers to integrate methods into tests.
• Explain quality, failure patterns, and uncertainty; guide model and product decisions.
• Validate automated graders against human ratings and identify biases and gaps.
• Share reusable methods and mentor peers across teams.

Skills

Leading complex evaluation projects

AI product evaluation

Designing human evaluations

Qualitative and quantitative research

Stakeholder management

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma is seeking a senior Researcher, AI Evaluations to join the AI Insights team. This individual contributor role focuses on conducting evaluation studies, building rubrics, datasets, and benchmarks to guide model and product improvements.

You’ll work with AI Research and Modeling Platform engineers to ensure repeatable evaluations and high-quality results. You’ll apply strong qualitative and quantitative methods to measure alignment, uncertainty, and bias, influencing cross-functional