Remote ML Evaluation Architect (Part-Time)

anyone-ai; United States; On-site; USD 83,000 - 124,000; Part time

Job summary

Anyone AI in the United States is seeking experienced Machine Learning Engineers to review and evaluate ML challenges used in model training. You will analyze experiments, datasets, metrics, and pipelines to determine technical soundness and reproducibility.

The role focuses on ensuring challenges reward good ML reasoning, not just brute-force tuning. Remote, part-time, project-based consulting, with a strong emphasis on written feedback and clear recommendations.

Qualifications

• 3+ years of hands-on applied ML experience.
• Strong ability to evaluate ML experiments and datasets.
• Ability to explain findings with written feedback.

Responsibilities

• Review ML challenges for design quality and solvability.
• Evaluate datasets for signals and learnability.
• Identify shortcuts and artifacts in synthetic data.
• Assess task calibration and difficulty.
• Review metrics and thresholds for improvements.
• Detect data leakage and metric gaming.
• Verify reproducibility across data → model → eval.
• Provide actionable recommendations to improve tasks.

Skills

Machine learning

Experiment design

Statistical testing

Data analysis

Written feedback

Tools

Python

TensorFlow

PyTorch

Jupyter

Job description

Anyone AI in the United States is seeking experienced Machine Learning Engineers to review and evaluate ML challenges used in model training. You will analyze experiments, datasets, metrics, and pipelines to determine technical soundness and reproducibility.

The role focuses on ensuring challenges reward good ML reasoning, not just brute-force tuning. Remote, part-time, project-based consulting, with a strong emphasis on written feedback and clear recommendations.