AI Evaluation Rubric Architect

ixolabs.ai; France; Sur place; EUR 60 000 - 84 000; Plein temps

Résumé du poste

IXO is seeking a seasoned reviewer and guideline engineer to translate complex program requirements into explicit rating criteria and edge-case examples.

You will formalize decision rules for AI outputs, resolve ambiguities, and produce revision rationales while maintaining consistency across related guideline sets. US-based contributors are preferred; about 35 hours per week with flexible scheduling.

Qualifications

• Three+ years of professional experience in linguistics, instructional design, technical writing or related field.
• Direct work on human-rater guidelines or rubrics in generative AI or RLHF.
• Ability to explain nuanced instructions clearly and resolve ambiguities.

Responsabilités

• Translate program requirements into explicit rating criteria and decision rules.
• Create edge-case examples and rating rubrics.
• Identify contradictions, vague terms and missing cases in guidelines.
• Test guidelines against difficult outputs and refine for clarity.
• Explain domain-specific distinctions for reviewers with different backgrounds.
• Produce documented changes showing revision improvements.

Connaissances

Linguistics

Instructional Design

Technical Writing

RLHF Guidelines

Description du poste

IXO is seeking a seasoned reviewer and guideline engineer to translate complex program requirements into explicit rating criteria and edge-case examples.

You will formalize decision rules for AI outputs, resolve ambiguities, and produce revision rationales while maintaining consistency across related guideline sets. US-based contributors are preferred; about 35 hours per week with flexible scheduling.