Remote AI Quality Evaluator - English Evaluation

AuraOne Human Data

United States

Remote

USD 34,000 - 55,000

Part time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

AuraOne Human Data is seeking an AI Quality Analyst - English evaluation track. You will review prompts and model outputs, compare responses, and provide structured feedback aligned with AuraOne's quality rubric.

As a remote contractor, you will label edge cases, annotate issues, and help calibrate scores weekly, enabling the modeling team to retrain effectively while maintaining high-quality standards for English evaluation tasks across batches.

Qualifications

  • Prior evaluation, annotation, or human-rater experience on ai quality analyst english evaluation or adjacent content.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and the rubric clause being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Evaluate ai quality analyst english evaluation model outputs against a versioned rubric and assign severity tags for AI Quality Analyst - English assignments.
  • Compare paired responses and pick the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them back to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
  • Document recurring failure modes so the modeling team can target them in the next training run.

Skills

Model output evaluation
Rubric-based annotation
Severity tagging
Inter-rater calibration
AI Quality Analyst English evaluation
Domain-Specific Languages

Job description

AuraOne Human Data is seeking an AI Quality Analyst - English evaluation track. You will review prompts and model outputs, compare responses, and provide structured feedback aligned with AuraOne's quality rubric.

As a remote contractor, you will label edge cases, annotate issues, and help calibrate scores weekly, enabling the modeling team to retrain effectively while maintaining high-quality standards for English evaluation tasks across batches.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Quality Analyst - English
AI Quality Analyst - English

AuraOne, Inc. • United States

Remote
USD 34,000 - 55,000
Remote English Language Evaluator for AI Rubric QA & Feedback
Remote English Language Evaluator for AI Rubric QA & Feedback

AuraOne Human Data • United States

Remote
USD 28,000 - 41,000
Remote AI Evaluation Annotator — English Creative Data
Remote AI Evaluation Annotator — English Creative Data

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000
Remote AI Conversation Quality Auditor
Remote AI Conversation Quality Auditor

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000
Linguistics background
Inter-rater metrics experience
Subject-matter domain knowledge
Remote AI Evaluation Annotator - English Creative Data
Remote AI Evaluation Annotator - English Creative Data

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000
Remote AI Data Evaluator, English & Assamese
Remote AI Data Evaluator, English & Assamese

Auraone • United States

Remote
USD 21,000 - 28,000
Remote work
Contractor flexibility
Remote AI Data Annotation Evaluator & Quality Reviewer
Remote AI Data Annotation Evaluator & Quality Reviewer

AuraOne, Inc. • United States

Remote
USD 28,000 - 41,000
Remote AI Evaluation & Feedback Specialist
Remote AI Evaluation & Feedback Specialist

Auraone • United States

Remote
USD 34,000 - 83,000
Remote work
Contractor engagement
US-eligible
Remote: Japanese AI Evaluation & Feedback Specialist
Remote: Japanese AI Evaluation & Feedback Specialist

Auraone • United States

Remote
USD 58,000 - 80,000
Remote English AI Quality Evaluator
Remote English AI Quality Evaluator

Anyone AI • United States

Remote
USD 9,600 - 17,000