ML Challenge Task Auditor

HumanitApp; Northern (KY); Hybrid; USD 96,000 - 124,000; Full time

Job summary

HumanitApp is seeking an evaluator to review the quality and rigor of applied ML tasks used to train and assess frontier AI models. You will examine experiment design, model-selection reasoning, and evaluation methodology, delivering clear rubric-based feedback to support research teams.

The role requires 3+ years in AI evaluation or related fields, with emphasis on rigorous methodological critique and precise written communication. Expect a 40-hour work week at a competitive hourly rate.

Qualifications

• 3+ years of experience in AI evaluation or related field.
• Ability to assess experiment designs and evaluation methodologies.
• Strong written communication for rubric-based feedback.

Responsibilities

• Evaluate quality, correctness, and methodological rigor of applied ML tasks used to train and evaluate models.
• Assess experiment design, model-selection reasoning, and evaluation methodology.
• Provide rubric-based written feedback to inform improvements and decision-making.

Skills

AI evaluation

Machine learning

Research

Job description

Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology

and provide clear, rubric-based written feedback.

Basic Qualifications • 3+ y...

AI evaluation Machine learning Research

Who this role may fit

This opportunity may suit professionals with relevant experience in AI evaluation, Machine learning, Research. Review the official description and requirements before applying.

Compensation context

The listing states $70 - $90 / hour. Confirm the final rate, workload, and payment terms during the official application process.

• You can show recent, specific work involving AI evaluation, Machine learning, Research.
• The listed commitment of 40 hours/week fits your schedule.
• You can communicate your reasoning clearly in the application language.
• You are comfortable with project availability and hours varying over time.

Not the right fit?

The listing states $70 - $90 / hour. Confirm the final rate, workload, and payment terms during the official application process.