Remote English Evaluation & Feedback Specialist

AI Trainer Jobs; United States; Remote; USD 83,000 - 138,000; Part time

Job summary

AuraOne is seeking an English Generalist to remotely evaluate prompts and responses against our quality rubric. You will compare paired outputs, label edge cases, and provide structured feedback to retrain the modeling team.

Strong English, attention to detail, and the ability to apply rubrics consistently are essential. This remote contractor role requires reliable 10+ hours per week and offers $20–$100 per hour depending on tasks.

Qualifications

• Prior evaluation, annotation, or human-rater experience on english generalist evaluation or adjacent content for English Generalist work.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Experience with inter-rater agreement metrics and calibration cycles.

Responsibilities

• Evaluate english generalist evaluation model outputs against a versioned rubric and assign severity tags for English Generalist assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

English Generalist evaluation

Advanced English proficiency

AP Style

Copyediting

language models

Linguistics

proofreading

Reading comprehension

Tools

Job description

AuraOne is seeking an English Generalist to remotely evaluate prompts and responses against our quality rubric. You will compare paired outputs, label edge cases, and provide structured feedback to retrain the modeling team.

Strong English, attention to detail, and the ability to apply rubrics consistently are essential. This remote contractor role requires reliable 10+ hours per week and offers $20–$100 per hour depending on tasks.