LLM Model Response Evaluation

Upwork; Maryland City (MD); Remote; USD 28,000 - 55,000; Full time

Job summary

Upwork is seeking experts to evaluate model-generated content against quality rubrics such as factuality, consistency, aesthetics, and other criteria. Tasks cover UI widgets, infographics, text, images, audio, video, HTML widgets, PDFs, and multiple modalities, across diverse topics.

Applicants should have 3+ years of hands-on GenAI evaluation experience, a Bachelor’s degree, and strong research skills to judge unfamiliar topics using trusted sources. Remote, flexible, with a variable workload.

Qualifications

• 3+ years of hands-on experience evaluating GenAI data.
• Bachelor's degree required.
• Ability to research unfamiliar topics from trusted sources.
• Experience evaluating multimodal content.

Responsibilities

• Evaluate UI widgets, infographics, image factuality, and side-by-side comparisons.
• Handle content across text, images, audio, video, HTML widgets, PDFs.
• Work across domains including arts, culture, history, science, engineering.
• Research unfamiliar topics using trusted sources before deciding.
• Follow detailed project guidelines in the evaluation platform.

Skills

GenAI evaluation

Research ability

Multimodal eval

3+ years exp

Education

Bachelor's Degree

Job description

Our client, a global technology company that helps businesses build, train, and manage AI systems is looking for experts to evaluate model-generated content against defined quality rubrics such as factuality, consistency, aesthetics, and other evaluation criteria.

Job Description

• Evaluating UI widgets, infographics, image factuality, side-by-side comparisons, and similar AI evaluation activities.
• The work may involve text, images, audio, video, HTML widgets, PDFs, or combinations of these modalities.
• The work is domain-agnostic and may cover topics across arts, culture, history, science, engineering, and more.
• Resources will be expected to independently research unfamiliar topics using trusted sources before making evaluation decisions.
• Each task will include detailed project guidelines within the evaluation platform.

Qualifications

• 3+ years of hands‑on experience in LLM / GenAI data evaluation.
• Bachelor's Degree required
• Ability to research unfamiliar topics using trusted sources and make well-supported judgments.
• Comfortable evaluating content across multiple modalities

Flexible and remote work

Variable workload:

Accept or decline tasks based on your availability

No guaranteed hours:

Workload may vary weekly