Senior AI Evaluation Researcher - Remote

AItoolnavio; San Francisco, New York (CA, NY); Remote; USD 169,000 - 250,000; Full time

Job summary

Figma is seeking a Researcher, AI Evaluations to join AI Insights within Figma’s Research team. This senior individual contributor role reports to the AI Insights lead and collaborates with AI Research within AI Platform.

You’ll strengthen evaluation methods through hands-on studies and reliable measures reflecting user needs. You’ll define quality criteria, build rubrics and benchmarks, validate automated graders against human ratings, and translate findings into model improvements that shape

Qualifications

• 7+ years of experience in research or related field and independently leading complex evaluation projects.
• Hands-on experience evaluating AI products and translating findings into model improvements.
• Expertise designing human evaluations, rubrics, datasets, and benchmarks.
• Strong qualitative and quantitative research skills to analyze data and interpret uncertainty.

Responsibilities

• Own evaluation research for priority AI workstreams; define quality criteria, design studies, analyze results, and recommend actions.
• Build rubrics, representative evaluation datasets, benchmarks, and quality thresholds reflecting user needs.
• Collaborate with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and tests.
• Explain quality, failure patterns, and uncertainty; advise actions guiding model selection and product priorities.
• Validate automated graders against human ratings; identify bias, gaps, and uncertainty.
• Bridge user research with AI Research’s model development; share methods and mentor peers.

Skills

Evaluation leadership

Qualitative analysis

Quantitative analysis

Stakeholder management

Experiment design

Data interpretation

Communication of findings

Education

Advanced training in quantitative discipline

Tools

Braintrust

LangSmith

DeepEval

Job description

Figma is seeking a Researcher, AI Evaluations to join AI Insights within Figma’s Research team. This senior individual contributor role reports to the AI Insights lead and collaborates with AI Research within AI Platform.

You’ll strengthen evaluation methods through hands-on studies and reliable measures reflecting user needs. You’ll define quality criteria, build rubrics and benchmarks, validate automated graders against human ratings, and translate findings into model improvements that shape