Senior Staff, Applied AI Research - Frontier Evaluations

Fdetimes; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity

On-site in SF

Job summary

Artificial Analysis in San Francisco seeks a Member of Technical Staff (Applied AI Research) to design and ship evaluations that set industry standards for AI measurement. You will build benchmarks, datasets and tooling, run evaluations on frontier models, and publish analyses that labs, enterprises and policymakers rely on.

You should have 3+ years in AI or ML, strong Python and data analysis skills, and a passion for frontier AI.

Qualifications

• 3+ years of relevant professional experience in AI/ML or research.

Responsibilities

• Design frontier evaluations: conceive, build and ship novel evaluation methodologies.
• Build evaluation datasets and infrastructure for large-scale benchmarking.
• Evaluate every major model across frontier releases and ensure result integrity.
• Publish influential analysis: reports, indexes and visualizations for labs and industry.
• Collaborate with frontier labs on pre-release models and loom with the research teams.

Skills

Python

Data analysis

Frontier AI

Job description

Artificial Analysis in San Francisco seeks a Member of Technical Staff (Applied AI Research) to design and ship evaluations that set industry standards for AI measurement. You will build benchmarks, datasets and tooling, run evaluations on frontier models, and publish analyses that labs, enterprises and policymakers rely on.

You should have 3+ years in AI or ML, strong Python and data analysis skills, and a passion for frontier AI.