Expert Problem Author, Computational AI Benchmarks

Toptal

United Kingdom

Remote

GBP 100,000 - 150,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Toptal is seeking domain experts to craft highly challenging, computer-solvable problems that advance frontier AI model evaluation. You will rely on deep field knowledge and rigorous problem design rather than code files.

You’ll ensure problems are solvable via Linux commands, with a single objective answer and robust edge cases. This remote role emphasizes precision and reproducibility across expert reviewers.

Qualifications

  • 8+ years of professional or research experience in your domain.
  • PhD-level depth in your field (strongly preferred).
  • Hands-on experience solving problems with computational tools.

Responsibilities

  • Design original, PhD-level problems within assigned domain or field.
  • Ensure each problem is solvable computationally on a Linux machine.
  • Define a single clear, objectively correct answer for automated evaluation.
  • Collaborate with reviewers to finalize problem statements and formats.

Skills

Domain expertise
Linux scripting
Research communication

Education

PhD in relevant field

Tools

Simulations
Numerical methods
Data analysis
Scripting

Job description

We're looking for experienced domain experts to design very hard, computer-solvable problems that today’s leading AI systems can't crack. Each problem you write becomes a benchmark task used to measure and improve frontier AI models. Your field knowledge is what matters here. This is not a software engineering role, and you won't write code files: once you submit, a technical reviewer turns your problem into the files needed to run it.

What You’ll Do

  • Design original, PhD-level problems within an assigned domain (and subdomain, where given), or in a field of your choice when none is assigned
  • Make sure each problem can only be solved computationally, by running commands or scripts on a Linux machine, and not by hand or with a calculator
  • Define one clear, objectively correct answer that a program can mark right or wrong, with no human judgment needed
  • Make problems robust enough that two independent experts would arrive at the same answer
  • Test problems against current state-of-the-art AI tools and refine them until those tools fail
  • Work with reviewers to clarify problem statements, edge cases and the expected answer format

What You Bring

  • 8+ years of professional or research experience in your domain
  • PhD-level depth in your field (a PhD is strongly preferred; equivalent senior research or industry expertise is considered)
  • Hands-on experience solving problems in your field with computational tools: simulations, numerical methods, data analysis, scripting or domain-specific software
  • Comfort working in a Linux environment at a practical level
  • A rigorous, precise approach: you can tell the difference between a hard problem and an ambiguous one
  • Strong written English for clear, unambiguous problem statements

Nice to Have

  • Experience building exams, qualifying problems, competition problems or research benchmarks
  • Prior work in AI training, model evaluation or red-teaming
  • Peer-reviewed publications or other recognized contributions in your field
  • Familiarity with how large language models and AI agents fail on technical tasks

Why This Role

  • Shape how the world’s most advanced AI systems are measured
  • Fully remote and flexible: work on your own schedule
  • Use deep expertise on intellectually demanding, original problems
  • Join a vetted network of senior experts across scientific and technical fields
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Operations & Finance Domain Expert — AI Benchmark Problem Author
Operations & Finance Domain Expert — AI Benchmark Problem Author

Toptal • United Kingdom

On-site
GBP 90,000 - 130,000
Senior Expert Problem Designer for AI Benchmarks
Senior Expert Problem Designer for AI Benchmarks

Toptal • United Kingdom

Remote
GBP 100,000 - 150,000
Cs Benchmark Expert - Fully Remote | Upto $84/Hr
Cs Benchmark Expert - Fully Remote | Upto $84/Hr

Mercor • United Kingdom

Remote
GBP 69,000 - 88,000
AI Math Problem Designer (Remote, Part-Time)
AI Math Problem Designer (Remote, Part-Time)

Anyone AI Inc. • United Kingdom

Remote
GBP 34,000 - 49,000
Remote work
Operations & Finance AI Benchmark Problem Architect
Operations & Finance AI Benchmark Problem Architect

Toptal • United Kingdom

On-site
GBP 90,000 - 130,000
Single-Cell Genomics Expert - Remote | Upto $100/Hr
Single-Cell Genomics Expert - Remote | Upto $100/Hr

Mercor • United Kingdom

Remote
GBP 73,000 - 104,000
Healthcare & Medical Expert
Healthcare & Medical Expert

Anyone AI • Greater London

Remote
GBP 31,000 - 62,000
Software Engineering Expert - Fully Remote | Up To $180/Hr
Software Engineering Expert - Fully Remote | Up To $180/Hr

Mercor • United Kingdom

Remote
GBP 125,000 - 188,000
Applied Physics
Applied Physics

Alignerr Corp. • City of Edinburgh

Remote
GBP 55,000 - 110,000
Fully remote
Biology Subject Matter Expert - AI Content Specialist
Biology Subject Matter Expert - AI Content Specialist

Alignerr Corp. • Cambridge

Remote
GBP 42,000 - 83,000
Remote work
Flexible hours
Contract-based