Senior Expert Problem Designer for AI Benchmarks

Toptal

United Kingdom

Remote

GBP 100,000 - 150,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Toptal is seeking domain experts to craft highly challenging, computer-solvable problems that advance frontier AI model evaluation. You will rely on deep field knowledge and rigorous problem design rather than code files.

You’ll ensure problems are solvable via Linux commands, with a single objective answer and robust edge cases. This remote role emphasizes precision and reproducibility across expert reviewers.

Qualifications

  • 8+ years of professional or research experience in your domain.
  • PhD-level depth in your field (strongly preferred).
  • Hands-on experience solving problems with computational tools.

Responsibilities

  • Design original, PhD-level problems within assigned domain or field.
  • Ensure each problem is solvable computationally on a Linux machine.
  • Define a single clear, objectively correct answer for automated evaluation.
  • Collaborate with reviewers to finalize problem statements and formats.

Skills

Domain expertise
Linux scripting
Research communication

Education

PhD in relevant field

Tools

Simulations
Numerical methods
Data analysis
Scripting

Job description

Toptal is seeking domain experts to craft highly challenging, computer-solvable problems that advance frontier AI model evaluation. You will rely on deep field knowledge and rigorous problem design rather than code files.

You’ll ensure problems are solvable via Linux commands, with a single objective answer and robust edge cases. This remote role emphasizes precision and reproducibility across expert reviewers.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Expert Problem Author, Computational AI Benchmarks
Expert Problem Author, Computational AI Benchmarks

Toptal • United Kingdom

Remote
GBP 100,000 - 150,000
Operations & Finance AI Benchmark Problem Architect
Operations & Finance AI Benchmark Problem Architect

Toptal • United Kingdom

On-site
GBP 90,000 - 130,000
AI Math Problem Designer (Remote, Part-Time)
AI Math Problem Designer (Remote, Part-Time)

Anyone AI Inc. • United Kingdom

Remote
GBP 34,000 - 49,000
Remote work
Cs Benchmark Expert - Fully Remote | Upto $84/Hr
Cs Benchmark Expert - Fully Remote | Upto $84/Hr

Mercor • United Kingdom

Remote
GBP 69,000 - 88,000
Staff AI Benchmark Engineer
Staff AI Benchmark Engineer

Stealth - AI x Bio • Greater London

On-site
GBP 90,000 - 150,000
Remote AI Training & Evaluation Engineer
Remote AI Training & Evaluation Engineer

Prolific • Manchester

Remote
GBP 37,000 - 61,000
Senior Software Engineer — AI Evaluation (Remote)
Senior Software Engineer — AI Evaluation (Remote)

Prolific • Wales

Remote
GBP 51,000 - 73,000
Software Engineering Expert - Fully Remote | Up To $180/Hr
Software Engineering Expert - Fully Remote | Up To $180/Hr

Mercor • United Kingdom

Remote
GBP 125,000 - 188,000
Senior AI Code Review Engineer Remote Contract
Senior AI Code Review Engineer Remote Contract

Mercor • United Kingdom

Remote
GBP 125,000 - 188,000
Biology AI Evaluation Specialist (Remote)
Biology AI Evaluation Specialist (Remote)

Alignerr Corp. • City of Edinburgh

Remote
GBP 28,000 - 55,000
Autonomy
Flexible hours
Global collaboration