Machine Learning Research Evaluation Expert

OpenTrain AI, Inc.; United States; Remote; USD 131,000 - 186,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Machine Learning Research Evaluation Expert to design and solve realistic ML problems, then provide reference answers that reveal flawed methods and weak conclusions. The work is remote, with a pay of $115 per hour and a minimum of 20 hours per week.

You will describe each reasoning step and focus on evaluation tasks within your specialty. Requirements include a master's or PhD in CS, statistics, or mathematics, plus at least three years of post-degree ML research

Qualifications

• 3+ years of full-time ML research after degree.
• Master's or PhD in CS, statistics, mathematics, or related field.
• Strong Python skills and experience with a DL framework (PyTorch or JAX).

Responsibilities

• Create and solve machine learning research problems and write reference answers.
• Explain the reasoning behind each step of your reference answers.
• Spot methodological mistakes and weak conclusions in research problems.
• Focus on evaluation work within your specialty.

Skills

Experiment design and ablations

Python

ML model evaluation

PyTorch or JAX

Education

Master's or PhD in CS/Statistics/Math

Tools

PyTorch

JAX

Job description

Machine Learning Research Evaluation Expert

Create and solve machine learning research problems, then write reference answers that reveal flawed methods and weak conclusions. This remote contractor role pays $115 per hour and requires at least 20 hours per week.

Remote Hourly · $115/hr

$115/hr

Compensation

Contract, part-time

Engagement

Remote

Location

Open in 95 countries

• Albania
• Algeria
• Antigua and Barbuda
• Armenia
• Australia
• Austria
• Bahamas
• Bahrain
• Belgium
• Benin
• Bhutan
• Bosnia and Herzegovina
• Botswana
• Brunei
• Bulgaria
• Canada
• Costa Rica
• Côte d’Ivoire
• Croatia
• Cyprus
• Czechia
• Denmark
• Ecuador
• El Salvador
• Estonia
• Ethiopia
• Finland
• France
• Gambia
• Germany
• Greece
• Guatemala
• Guyana
• Hong Kong
• Hungary
• Iceland
• India
• Indonesia
• Ireland
• Israel
• Italy
• Jamaica
• Jordan
• Kenya
• Kuwait
• Latvia
• Liechtenstein
• Lithuania
• Luxembourg
• Madagascar
• Malaysia
• Malta
• Mauritius
• Mexico
• Moldova
• Mongolia
• Morocco
• Mozambique
• Namibia
• Netherlands
• New Zealand
• North Macedonia
• Norway
• Oman
• Pakistan
• Panama
• Peru
• Philippines
• Poland
• Portugal
• Qatar
• Romania
• Rwanda
• Saint Lucia
• Senegal
• Serbia
• Singapore
• Slovakia
• Slovenia
• South Africa
• Spain
• Sri Lanka
• Sweden
• Switzerland
• Taiwan
• Tanzania
• Thailand
• Trinidad and Tobago
• Tunisia
• Türkiye
• United Arab Emirates
• United Kingdom
• United States
• Uzbekistan
• Vietnam

The work

Create realistic machine learning research problems based on your research expertise, then solve them and write clear reference answers. Your work helps evaluate AI models by showing the difference between sound research judgment and answers that seem plausible but are wrong.

• Design problems about experiment design, ablations, model evaluation, and failure analysis.
• Explain the reasoning behind each step of your reference answers.
• Spot methodological mistakes and weak research conclusions that may take practical experience to recognize.
• Focus on evaluation work in your research specialty rather than completing a high volume of tasks.

What it pays and takes

• Pay: $115 per hour, paid weekly via Stripe.
• Schedule: At least 20 hours per week. The role is asynchronous with no fixed hours.
• Work arrangement: Remote contractor role, open to candidates based in eligible countries.
• Language: Clear written English and the ability to explain technical results step by step.
• Education: A master's or PhD in computer science, statistics, mathematics, or a related field.
• Experience: At least three years of full-time machine learning research after completing your degree. Postdoctoral research counts; doctoral study years do not.
• Skills: Hands-on experience designing experiments, running ablations, or evaluating machine learning models; strong Python skills; and experience with a deep learning framework such as PyTorch or JAX.
• Helpful experience: Publications at venues such as NeurIPS, ICML, ICLR, ACL, or CVPR, or work in an industry research lab or university research group.

About AI training work

AI training work is the human work behind building and evaluating AI systems, including writing and reviewing examples that help measure model performance. Research expertise matters for this project because the evaluation problems need to distinguish sound methods from convincing mistakes.