Remote LLM Multimodal Evaluation Specialist

Upwork; Maryland City (MD); Remote; USD 28,000 - 55,000; Full time

Job summary

Upwork is seeking experts to evaluate model-generated content against quality rubrics such as factuality, consistency, aesthetics, and other criteria. Tasks cover UI widgets, infographics, text, images, audio, video, HTML widgets, PDFs, and multiple modalities, across diverse topics.

Applicants should have 3+ years of hands-on GenAI evaluation experience, a Bachelor’s degree, and strong research skills to judge unfamiliar topics using trusted sources. Remote, flexible, with a variable workload.

Qualifications

• 3+ years of hands-on experience evaluating GenAI data.
• Bachelor's degree required.
• Ability to research unfamiliar topics from trusted sources.
• Experience evaluating multimodal content.

Responsibilities

• Evaluate UI widgets, infographics, image factuality, and side-by-side comparisons.
• Handle content across text, images, audio, video, HTML widgets, PDFs.
• Work across domains including arts, culture, history, science, engineering.
• Research unfamiliar topics using trusted sources before deciding.
• Follow detailed project guidelines in the evaluation platform.

Skills

GenAI evaluation

Research ability

Multimodal eval

3+ years exp

Education

Bachelor's Degree

Job description

Upwork is seeking experts to evaluate model-generated content against quality rubrics such as factuality, consistency, aesthetics, and other criteria. Tasks cover UI widgets, infographics, text, images, audio, video, HTML widgets, PDFs, and multiple modalities, across diverse topics.

Applicants should have 3+ years of hands-on GenAI evaluation experience, a Bachelor’s degree, and strong research skills to judge unfamiliar topics using trusted sources. Remote, flexible, with a variable workload.