Memory Evaluation QA Specialist - Remote Contractor

AuraOne; United States; On-site; USD 41,328 - 68,880; Full time

Job summary

AuraOne is seeking a Memory Evaluation Model Evaluation Specialist for a remote, contract engagement. You will review prompts and model outputs, apply the quality rubric, and provide structured feedback to help retrain the evaluation system.

You will compare paired responses, label edge cases, and document rationale and failure modes to inform improvements. Strong attention to detail and clear written reasoning are essential.

Qualifications

• Prior evaluation, annotation, or human-rater experience on memory evaluation model evaluation evaluation or adjacent content for Memory Evaluation Model Evaluation Specialist work.

Responsibilities

• Evaluate memory evaluation model evaluation model outputs against a versioned rubric and assign severity tags for Memory Evaluation Model Evaluation Specialist assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Memory Evaluation Model Evaluation

Frontier evaluation

Rubric calibration

Failure analysis

Memory

Job description

AuraOne is seeking a Memory Evaluation Model Evaluation Specialist for a remote, contract engagement. You will review prompts and model outputs, apply the quality rubric, and provide structured feedback to help retrain the evaluation system.

You will compare paired responses, label edge cases, and document rationale and failure modes to inform improvements. Strong attention to detail and clear written reasoning are essential.