Applied Machine Learning Task Auditor

OpenTrain AI; Northern (KY); Hybrid; USD 96,000 - 124,000; Part time

Job summary

OpenTrain AI is recruiting an Applied Machine Learning Task Auditor to evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train frontier AI models. The work centers on evaluation rather than development.

You will determine whether experiments are designed soundly, model-selection reasoning is supported by evidence, and evaluation methods produce trustworthy conclusions.

Qualifications

• Hands-on applied or experimental machine-learning experience (3+ years).
• Experience with experiment design, model selection, and hyperparameter tuning.
• Strong data-quality controls and data leakage detection.
• Knowledge of train/test and cross-validation hygiene.
• Proficiency with PyTorch, TensorFlow, scikit-learn, and XGBoost.

Responsibilities

• Review applied machine-learning tasks for methodological rigor and evidentiary support.
• Assess hyperparameter-tuning choices and evaluation methodology.
• Check for data leakage, metric gaming, and train/test hygiene issues.
• Provide clear written feedback using structured rubrics.
• Reproduce results when necessary.

Job description

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the #1 platform for finding and building careers in AI training and data labeling, helping contributors discover projects, build a professional profile, and apply in minutes.

• Remote contractor opportunity for candidates in the United States
• OpenTrain account creation is free
• Build a portfolio around credible AI training and evaluation experience

About AI Training and Model Evaluation

AI training is the human side of building artificial intelligence. People with technical expertise help prepare, review, and evaluate the examples and outputs used to improve modern AI systems. In this role, your analysis will support trustworthy conclusions about applied machine-learning experiments and model performance.

• Contribute to cutting-edge AI development through structured technical review
• Use specialized machine-learning knowledge to assess evidence and methodology
• Work remotely with a flexible contractor schedule

The Role

OpenTrain is recruiting an Applied Machine Learning Task Auditor to evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate frontier AI models. The work centers on applied and experimental machine-learning review, rather than LLM application development or MLOps.

You will determine whether experiments are designed soundly, model-selection reasoning is supported by evidence, and evaluation methods produce trustworthy conclusions. The role requires careful technical judgment and clear written communication through structured rubrics.

• Role focus: Applied Machine Learning Evaluation
• Data type: Text
• Primary task type: Evaluation and rating
• Experience level listed for the role: Entry level

What You'll Do

You will assess applied machine-learning tasks for methodological rigor, correctness, and evidentiary support. Your reviews may require reproducing results and explaining weaknesses or strengths in a clear, structured format.

• Review experiment design and model-selection decisions
• Assess hyperparameter-tuning choices and evaluation methodology
• Check for data leakage, metric gaming, and train/test or cross-validation hygiene issues
• Critique machine-learning claims against the available evidence
• Reproduce results when necessary
• Provide clear written feedback using structured rubrics

Required Qualifications

You should have at least three years of hands-on applied or experimental machine-learning experience. Your background must include experiment design, model selection, hyperparameter tuning, and evaluation methodology, along with strong data-quality controls.

• Hands-on applied or experimental machine-learning experience
• Experience with experiment design, model selection, and hyperparameter tuning
• Strong understanding of data leakage detection and metric gaming
• Knowledge of train/test and cross-validation hygiene
• Proficiency with PyTorch, TensorFlow, scikit-learn, and XGBoost
• Ability to evaluate machine-learning claims against evidence and reproduce results

Helpful Background

The following experience is valuable but presented as helpful background rather than a required qualification.

• Competition or benchmark experience, including Kaggle participation
• Graduate research in applied machine learning
• A publication record in applied machine learning
• Previous task-grading or peer-review experience
• Other structured technical assessment experience

Schedule, Location, and Pay

This is a remote contractor role for candidates located in the United States. The structured opportunity details call for 20 or more hours per week, with a default commitment of 40 hours per week.

• Location: United States
• Work arrangement: Remote
• Employment type: Contractor and part-time
• Time requirement: 20+ hours per week
• Default commitment: 40 hours per week
• Pay: $70-$90 per hour
• Language: English

Why Build Your AI Training Career With OpenTrain

AI training and data-labeling work is a fast-growing way to participate in technology without leaving remote flexibility behind. OpenTrain gives contributors one place to manage opportunities, show credible experience, and develop a durable portfolio in the field.

A stronger OpenTrain profile can help you present your technical background, discover projects aligned with your skills, and grow your work in AI training and evaluation over time.

• Find and apply for AI training opportunities in one place
• Build a professional profile and portfolio you control
• Turn specialized evaluation work into lasting career evidence