AI Benchmarking Expert - Complex Science Problems

Truelancer AI; United States; Remote; USD 83,000 - 152,000; Part time

Job summary

Truelancer AI is seeking Science Benchmarking Experts to design and evaluate high‑complexity scientific problems for AI benchmarking projects. You will craft multi‑step problems with structured solutions, document assumptions and edge cases, and estimate effort for expert solvers.

This is a remote, project‑based (1099) engagement in the United States. Domains span Physics to Computer Science, including Economics, Mathematics, and Neuroscience; diverse expertise highly valued.

Qualifications

• Design sophisticated problems requiring domain expertise and multi-step reasoning.
• Create problems needing hours or more for a qualified expert to solve.
• Develop structured solution approaches with clear assumptions and constraints.
• Explain why problems are challenging and computationally intensive.
• Estimate effort and review problems for ambiguity and uniqueness.

Responsibilities

• Design highly complex problem statements within your area of expertise.
• Create problems requiring several hours or more for a qualified human expert to solve.
• Develop a structured and logically sound solution approach for each problem.
• Clearly identify assumptions, constraints, edge cases, and other details.
• Explain why the problem is challenging or computationally/reasoning intensive.
• Estimate the approximate effort required for a qualified domain expert to solve the problem.
• Review problems for ambiguity, correctness, consistency, and uniqueness of the expected answer.
• Adapt problem statements based on benchmarking requirements.
• Evaluate problems using available AI tools/models and determine whether AI systems can solve them successfully.
• Identify where and why AI reasoning fails when it does.

Skills

Complex thinking

Problem design

Structured problem solving

Communication

AI tooling familiarity

Adaptability

Education

PhD preferred

Master's degree possible

Equivalent indicators of deep domain expertise

Tools

AI Tools

Job description

Truelancer AI is seeking Science Benchmarking Experts to design and evaluate high‑complexity scientific problems for AI benchmarking projects. You will craft multi‑step problems with structured solutions, document assumptions and edge cases, and estimate effort for expert solvers.

This is a remote, project‑based (1099) engagement in the United States. Domains span Physics to Computer Science, including Economics, Mathematics, and Neuroscience; diverse expertise highly valued.