ML Challenge Task Auditor

DigiNo; Northern (KY); Hybrid; USD 100,000 - 180,000; Full time

Job summary

Qualifications

• 3+ years hands-on applied ML, including experiment design and evaluation.
• Strong data-quality hygiene: leakage detection and train/test/CV rigor.
• Proficiency with PyTorch, TensorFlow, scikit-learn, XGBoost.
• Ability to critique ML claims and reproduce results.

Responsibilities

• Evaluate quality, correctness, and rigor of applied ML tasks used to train and evaluate models.
• Assess experiment design, model-selection reasoning, and evaluation methodology.
• Provide rubric-based written feedback outlining strengths and gaps.

Skills

Applied ML

Experiment design

Model selection

Evaluation methodology

Data quality

ML frameworks

Tools

PyTorch

TensorFlow

scikit-learn

XGBoost

Job description

Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology — and provide clear, rubric-based written feedback.

Basic Qualifications

• 3+ years hands-on applied/experimental ML (experiment design, model selection, hyperparameter tuning, evaluation methodology)
• Strong grasp of data-quality rigor: leakage detection, metric gaming, and train/test/CV hygiene
• Proficiency with standard ML frameworks (PyTorch, TensorFlow, scikit-learn, XGBoost)
• Ability to critique ML claims against evidence and reproduce results

Preferred Qualifications

• Competition / benchmark experience (e.g., Kaggle)
• Graduate research or publication record in applied ML
• Prior task-grading or peer-review experience

Note: this role evaluates applied/experimental ML rigor — it is not an LLM-application-building or MLOps role.