[India] AI Science Benchmarking Expert – Complex Problem Creation

Truelancer AI; United States; Remote; USD 110,000 - 165,000; Part time

Benefits offered by this job

Remote freelance project

Job summary

Truelancer AI is seeking domain experts, researchers, and PhDs to contribute to an advanced AI Benchmarking Project, designing and evaluating high-complexity scientific and technical problems that require hours to solve.

As a Science Benchmarking Expert, you will craft challenging problems, develop structured solution approaches, identify assumptions and edge cases, and help assess whether AI systems can solve them in remote, project-based freelance engagements.

Qualifications

• PhD preferred in a relevant scientific, technical, mathematical, or quantitative discipline.
• Master's degree holders with strong relevant expertise may also be considered.
• Equivalent indicators of deep domain expertise may be considered for exceptional candidates.

Responsibilities

• Design highly complex, challenging problem statements within your area of expertise.
• Create problems that require several hours or more for a qualified human expert to solve.
• Develop a structured and logically sound solution approach for each problem.
• Clearly identify assumptions, constraints, edge cases, and other details required for a correct solution.
• Explain why the problem is challenging or computationally/reasoning intensive.
• Estimate the approximate effort required for a qualified domain expert to solve the problem.
• Review problems for ambiguity, correctness, consistency, and uniqueness of the expected answer.
• Adapt problem statements based on project-specific benchmarking requirements.
• Evaluate problems using available AI tools/models and determine whether AI systems can solve them successfully.
• When an AI system fails, identify where and why the reasoning fails.
• Help distinguish between problems that are genuinely difficult and problems that are simply poorly defined.

Skills

Complex Thinking & Problem Design

Communication

Structured Problem Solving

Attention to Detail

Adaptability

AI Familiarity

Education

PhD preferred

Master's degree with strong expertise

Equivalent indicators of deep domain expertise

Job description

About the Project

Truelancer AI is looking for exceptional domain experts, researchers, and PhDs to contribute to an advanced AI Benchmarking Project.

The project focuses on creating and evaluating high-complexity scientific and technical problems that can be used to assess the reasoning capabilities of modern AI systems.

You will use your subject-matter expertise to design problems that require several hours or more for a human expert to solve, develop structured solution approaches, and evaluate whether current AI systems can successfully solve them.

This is an opportunity to work at the intersection of deep domain expertise, complex problem solving, and Artificial Intelligence evaluation.

Engagement Type - Project-based / Freelance Contract

Work Mode - Remote

Locations - Inda (CST & EST preferred)

Domains

We are particularly interested in experts across the following domains:

• Physics

• Chemistry

• Materials Science

• Environmental Science

• Molecular Biology

• Genetics

• Neuroscience

• Mathematics

• Economics

• Computer Science

• Geology

• Geophysics

• And other advanced scientific or technical disciplines

Experts from additional specialized domains are also encouraged to apply. Greater diversity of subject expertise is valuable for this project.

What You Will Do

As a Science Benchmarking Expert, you will:

• Design highly complex, challenging problem statements within your area of expertise.

• Create problems that require several hours or more for a qualified human expert to solve.

• Develop a structured and logically sound solution approach for each problem.

• Clearly identify assumptions, constraints, edge cases, and other details required for a correct solution.

• Explain why the problem is challenging or computationally/reasoning intensive.

• Estimate the approximate effort required for a qualified domain expert to solve the problem.

• Review problems for ambiguity, correctness, consistency, and uniqueness of the expected answer.

• Adapt problem statements based on project-specific benchmarking requirements.

• Evaluate problems using available AI tools/models and determine whether AI systems can solve them successfully.

• When an AI system fails, identify where and why the reasoning fails.

• Help distinguish between problems that are genuinely difficult and problems that are simply poorly defined.

Key Skills Required

1. Complex Thinking & Problem Design

Ability to formulate sophisticated problems that require substantial domain expertise, reasoning, analysis, or multi-step problem solving.

2. Communication

Ability to explain complex scientific or technical concepts in a clear and simple manner, including why a particular problem is difficult.

3. Structured Problem Solving

Ability to break down complex problems into logically structured steps and develop a defensible solution methodology.

4. Attention to Detail

Strong attention to detail, including the ability to:

• Identify hidden assumptions

• Define constraints explicitly

• Detect inconsistencies

• Validate calculations and reasoning

• Identify edge cases

• Ensure the problem has a well-defined expected outcome

5. Adaptability

Ability to apply your domain expertise to different project requirements and modify problem statements to meet specific benchmarking criteria.

For example, you may need to modify a subjective problem so that it has a clearly defensible or uniquely correct answer.

6. AI Familiarity

Basic familiarity with modern AI tools and LLMs.

You should be able to:

• Determine whether an AI system can solve a particular problem.

• Test complex problems using AI tools.

• Identify incorrect, incomplete, or hallucinated AI responses.

• Explain where an AI system's reasoning breaks down.

• Distinguish between an AI failure caused by lack of knowledge and one caused by reasoning or problem-solving limitations.

Eligibility

Education

• PhD preferred, particularly in a relevant scientific, technical, mathematical, or quantitative discipline.

• Master's degree holders with strong relevant expertise and demonstrable experience may also be considered.

• Equivalent indicators of deep domain expertise may be considered for exceptional candidates.

Experience

No specific number of years of experience is required.

We are primarily interested in the depth of expertise, quality of thinking, and ability to create genuinely challenging problems.

Candidates should be able to demonstrate their expertise through academic, research, professional, or other relevant work.