Remote Evaluation Specialist: English & Telugu

Auraone; United States; Remote; USD 21,000 - 28,000; Full time

Job summary

AI Trainer Jobs seeks a Generalist - English & Telugu evaluator to review prompts and responses from AuraOne's data-labeling rubric. You will compare outputs, label edge cases, and provide structured feedback for model retraining.

Applicants should have prior evaluation experience, strong reasoning, and be available remotely for at least 10 hours per week. This is a contractor role with flexible pacing.

Qualifications

• Prior evaluation, annotation, or human-rater experience on generalist English–Telugu content.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning naming the issue and rubric clause.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger with rationale.
• Label hallucinations, instruction-following failures, and unsafe content.
• Capture ambiguous prompts and route to program team for rubric updates.
• Calibrate reviewer-quality scores against gold-standard examples weekly.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Generalist English Telugu evaluation

Job description

AI Trainer Jobs seeks a Generalist - English & Telugu evaluator to review prompts and responses from AuraOne's data-labeling rubric. You will compare outputs, label edge cases, and provide structured feedback for model retraining.

Applicants should have prior evaluation experience, strong reasoning, and be available remotely for at least 10 hours per week. This is a contractor role with flexible pacing.