[USA] AI Science Benchmarking Expert – Complex Problem Creation

Truelancer AI; United States; Remote; USD 83,000 - 152,000; Part time

Job summary

Truelancer AI is seeking Science Benchmarking Experts to design and evaluate high‑complexity scientific problems for AI benchmarking projects. You will craft multi‑step problems with structured solutions, document assumptions and edge cases, and estimate effort for expert solvers.

This is a remote, project‑based (1099) engagement in the United States. Domains span Physics to Computer Science, including Economics, Mathematics, and Neuroscience; diverse expertise highly valued.

Qualifications

• Design sophisticated problems requiring domain expertise and multi-step reasoning.
• Create problems needing hours or more for a qualified expert to solve.
• Develop structured solution approaches with clear assumptions and constraints.
• Explain why problems are challenging and computationally intensive.
• Estimate effort and review problems for ambiguity and uniqueness.

Responsibilities

• Design highly complex problem statements within your area of expertise.
• Create problems requiring several hours or more for a qualified human expert to solve.
• Develop a structured and logically sound solution approach for each problem.
• Clearly identify assumptions, constraints, edge cases, and other details.
• Explain why the problem is challenging or computationally/reasoning intensive.
• Estimate the approximate effort required for a qualified domain expert to solve the problem.
• Review problems for ambiguity, correctness, consistency, and uniqueness of the expected answer.
• Adapt problem statements based on benchmarking requirements.
• Evaluate problems using available AI tools/models and determine whether AI systems can solve them successfully.
• Identify where and why AI reasoning fails when it does.

Skills

Complex thinking

Problem design

Structured problem solving

Communication

AI tooling familiarity

Adaptability

Education

PhD preferred

Master's degree possible

Equivalent indicators of deep domain expertise

Tools

AI Tools

Job description

About the Project

Truelancer AI is looking for exceptional domain experts, researchers, and PhDs to contribute to an advanced AI Benchmarking Project.

The project focuses on creating and evaluating high-complexity scientific and technical problems that can be used to assess the reasoning capabilities of modern AI systems.

You will use your subject-matter expertise to design problems that require several hours or more for a human expert to solve, develop structured solution approaches, and evaluate whether current AI systems can successfully solve them.

This is an opportunity to work at the intersection of deep domain expertise, complex problem solving, and Artificial Intelligence evaluation.

Engagement Type - Project-based / 1099 Contract

Work Mode - Remote

Locations - United States (CST & EST preferred)

Domains

We are particularly interested in experts across the following domains:

• Physics

• Chemistry

• Materials Science

• Environmental Science

• Molecular Biology

• Genetics

• Neuroscience

• Mathematics

• Economics

• Computer Science

• Geology

• Geophysics

• And other advanced scientific or technical disciplines

Experts from additional specialized domains are also encouraged to apply. Greater diversity of subject expertise is valuable for this project.

What You Will Do

As a Science Benchmarking Expert, you will:

• Design highly complex, challenging problem statements within your area of expertise.

• Create problems that require several hours or more for a qualified human expert to solve.

• Develop a structured and logically sound solution approach for each problem.

• Clearly identify assumptions, constraints, edge cases, and other details required for a correct solution.

• Explain why the problem is challenging or computationally/reasoning intensive.

• Estimate the approximate effort required for a qualified domain expert to solve the problem.

• Review problems for ambiguity, correctness, consistency, and uniqueness of the expected answer.

• Adapt problem statements based on project-specific benchmarking requirements.

• Evaluate problems using available AI tools/models and determine whether AI systems can solve them successfully.

• When an AI system fails, identify where and why the reasoning fails.

• Help distinguish between problems that are genuinely difficult and problems that are simply poorly defined.

Key Skills Required

1. Complex Thinking & Problem Design

Ability to formulate sophisticated problems that require substantial domain expertise, reasoning, analysis, or multi‑step problem solving.

2. Communication

Ability to explain complex scientific or technical concepts in a clear and simple manner, including why a particular problem is difficult.

3. Structured Problem Solving

Ability to break down complex problems into logically structured steps and develop a defensible solution methodology.

4. Attention to Detail

Strong attention to detail, including the ability to:

• Identify hidden assumptions

• Define constraints explicitly

• Detect inconsistencies

• Validate calculations and reasoning

• Identify edge cases

• Ensure the problem has a well‑defined expected outcome

5. Adaptability

Ability to apply your domain expertise to different project requirements and modify problem statements to meet specific benchmarking criteria.

For example, you may need to modify a subjective problem so that it has a clearly defensible or uniquely correct answer.

6. AI Familiarity

Basic familiarity with modern AI tools and LLMs.

You should be able to:

• Determine whether an AI system can solve a particular problem.

• Test complex problems using AI tools.

• Identify incorrect, incomplete, or hallucinated AI responses.

• Explain where an AI system's reasoning breaks down.

• Distinguish between an AI failure caused by lack of knowledge and one caused by reasoning or problem‑solving limitations.

Eligibility

Education

• PhD preferred, particularly in a relevant scientific, technical, mathematical, or quantitative discipline.

• Master's degree holders with strong relevant expertise and demonstrable experience may also be considered.

• Equivalent indicators of deep domain expertise may be considered for exceptional candidates.

Experience

No specific number of years of experience is required.

We are primarily interested in the depth of expertise, quality of thinking, and ability to create genuinely challenging problems.

Candidates should be able to demonstrate their expertise through academic, research, professional, or other relevant work.

Guidelines

Truelancer AI verifies each project before posting. Apply on Truelancer AI; never pay to get work.