Qualitative Evaluation Engineer, Multimodal AI

Luma AI; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Luma AI is seeking a Qualitative Evaluation Engineer to design evaluative systems that capture believability, identity retention, and scene coherence in generative models. You’ll translate fuzzy quality into concrete criteria and collaborate with researchers and technical artists to guide model development.

You will lead qualitative studies, build scalable evaluation frameworks, and turn nuanced judgments into actionable insights for fine-tuning, data curation, and UX improvements.

Qualifications

• 5+ years in product evaluation, UX research, model testing, or similar structured qualitative assessment.
• Master's or higher in Cognitive Science, HCI, Design Research, Psychology, Media Studies, or related field.
• Deep familiarity with creative workflows for generative models (animation, filmmaking, digital art, VFX).
• Systems thinking: define abstract qualities like believability in clear evaluative terms.
• Excellent written communication and ability to synthesize nuanced judgment into actionable insight.

Responsibilities

• Evaluate generative model performance across diverse tasks, prompts, and modalities; surface failure modes and edge cases that hurt product quality.
• Build and maintain qualitative evaluation frameworks that are scalable and reusable.
• Translate high-level product goals into concrete evaluative criteria.
• Lead qualitative studies, side-by-side comparisons, and human-in-the-loop evaluations.
• Turn nuanced judgments into clear feedback informing fine-tuning, dataset curation, and product UX.

Skills

Qualitative evaluation

UX research

Model testing

Cross-functional collaboration

Written communication

Engineer–creative collaboration

Education

Master's degree in cognitive science, HCI, design research, psychology, or related field

Job description

Luma AI is seeking a Qualitative Evaluation Engineer to design evaluative systems that capture believability, identity retention, and scene coherence in generative models. You’ll translate fuzzy quality into concrete criteria and collaborate with researchers and technical artists to guide model development.

You will lead qualitative studies, build scalable evaluation frameworks, and turn nuanced judgments into actionable insights for fine-tuning, data curation, and UX improvements.