ML Challenge Auditor — Independent Contractor

Meridial; United States; Remote; USD 110,000 - 248,000; Full time

Job summary

Meridial seeks experienced technical experts to contribute to an independent contractor opportunity focused on auditing tasks used to train and evaluate AI systems. Experts will be matched to one specialty based on their experience from eight areas listed.

A technical assessment related to the area of expertise will be part of the selection process. Experts will assess the accuracy, realism, solvability, reproducibility, and test support of tasks, and provide clear, actionable feedback on issues

Qualifications

• Experience auditing tasks used to train and evaluate AI systems.
• Willingness to complete a technical assessment related to your area of expertise as part of selection.

Responsibilities

• Assess whether technical tasks are accurate, realistic, solvable, reproducible and supported by tests and evaluation criteria.
• Provide clear, actionable feedback on issues identified during audits.

Skills

GPU kernels

AWS Trainium/NKI

CVE & app security

Machine learning

AI-assisted dev workflows

AWS Serverless & IaC

Kubernetes

SWE-Bench

Tools

Kubernetes

Job description

Meridial seeks experienced technical experts to contribute to an independent contractor opportunity focused on auditing tasks used to train and evaluate AI systems. Experts will be matched to one specialty based on their experience from eight areas listed.

A technical assessment related to the area of expertise will be part of the selection process. Experts will assess the accuracy, realism, solvability, reproducibility, and test support of tasks, and provide clear, actionable feedback on issues