Remote PDF Annotation & Transcription Evaluator (Malayalam)

AuraOne; United States; On-site; USD 17,000 - 21,000; Part time

Job summary

Candidates should have prior evaluation or annotation experience, be detail-focused, and comfortable applying long rubrics across large batches with at least 10 hours per week available. This is a remote US-eligible contractor role.

Qualifications

• Prior evaluation, annotation, or human-rater experience on pdf annotation transcription Malayalam evaluation or adjacent content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate pdf annotation transcription experts malayalam evaluation outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Document recurring failure modes and escalate for rubric updates.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Malayalam evaluation

Job description

Candidates should have prior evaluation or annotation experience, be detail-focused, and comfortable applying long rubrics across large batches with at least 10 hours per week available. This is a remote US-eligible contractor role.