Remote Part-Time ML Engineer — Evaluation & Experiments

OpenTrain AI; United States; On-site; USD 82,656 - 123,984; Part time

Job summary

OpenTrain AI is seeking an experienced machine learning practitioner to act as a ground-truth expert for frontier model evaluation. You will turn research ideas into reproducible evaluation tasks, run training experiments, and analyze results to show correct solutions and where models fall short.

This is a contract, part-time position for US-based contributors, expected at 20+ hours per week.

Qualifications

• MSc/PhD in ML, CS or related STEM, or equivalent experience.
• At least 1 year in research or research-engineering role.
• Hands-on experience training and evaluating ML models end-to-end.
• Strong familiarity with large language models, their capabilities, limitations, and evaluation techniques.
• Proficient in Python and Git; comfortable in scripting and notebook environments.

Responsibilities

• Design, implement, and run end-to-end evaluation and experimentation workflows that benchmark language models and training approaches.
• Turn vague ML research ideas into well-defined, multi-step evaluation tasks and benchmarks.
• Implement changes, run training experiments, and analyze outcomes to demonstrate correct behavior.
• Build tasks and analyses around reinforcement learning concepts such as reward functions and training behavior.
• Evaluate frontier models on your tasks, document failure modes, and explain where and why they fall short.
• Collaborate with researchers and other experts to keep tasks consistent, rigorous, and reproducible.

Skills

Research problem solving

Written communication

Independent work

Experiment design

Education

MSc or PhD in ML/CS or related STEM (or equivalent experience)

1+ years in research or research-engineering role

Tools

Python

Git

Job description

OpenTrain AI is seeking an experienced machine learning practitioner to act as a ground-truth expert for frontier model evaluation. You will turn research ideas into reproducible evaluation tasks, run training experiments, and analyze results to show correct solutions and where models fall short.

This is a contract, part-time position for US-based contributors, expected at 20+ hours per week.