Remote Dialect Code-Switching AI Evaluator

AI Trainer Jobs; United States; Remote; USD 28,000 - 55,000; Part time

Job summary

AI Trainer Jobs is seeking a Dialect Code-Switching AI Evaluator to remotely review prompts and model outputs. You will compare paired responses, label edge cases, and provide structured feedback for retraining; this role focuses on dialect code-switching evaluation and quality control in a US-eligible remote setting.

The contractor position requires strong attention to detail, ability to apply rubrics consistently, and clear written rationales.

Qualifications

• Experience evaluating prompts and responses.
• Ability to apply rubrics consistently across batches.
• Strong written reasoning to name issues and rubric clauses.

Responsibilities

• Evaluate dialect code switching AI outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route to rubric updates.
• Calibrate reviewer scores against gold-standard examples weekly.

Skills

Dialect Code Switching

Annotation experience

Inter-rater calibration

Quality rubric

Job description

AI Trainer Jobs is seeking a Dialect Code-Switching AI Evaluator to remotely review prompts and model outputs. You will compare paired responses, label edge cases, and provide structured feedback for retraining; this role focuses on dialect code-switching evaluation and quality control in a US-eligible remote setting.

The contractor position requires strong attention to detail, ability to apply rubrics consistently, and clear written rationales.