AI Quality Evaluator — Remote, Flexible Hours

Outlier

Pittsburgh (Allegheny County)

Remote

USD 15,000 - 25,000

Full time

14 hours ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible hours

Free LLM access

Weekly payments

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, compare generated images and videos, and explain in writing what looked or sounded off, all remotely and on your schedule.

At Outlier, your feedback helps shape how AI systems reason and learn. You can work from home or while traveling, earn up to $18 USD per hour, and enjoy weekly payments and free access to leading LLMs.

Qualifications

Fluent English speaker with strong written communication and comfortable with natural everyday conversational speech.
Excellent attention to detail and patience to replay or zoom in when something seems off.
Nice to have: pursuing or holding a Bachelor's degree or above.
Comfortable thinking on your feet and explaining decisions clearly in writing.

Responsibilities

Holding live, scenario-based conversations with AI voice models and ranking naturalness, usefulness, and clarity.
Comparing AI-generated images and videos and judging which follows the prompt better.
Inspecting outputs closely for issues like distorted anatomy, flickering, or audio-video mismatch, then explaining findings in English.
Rating each output across dimensions and providing a concise, evidence-based explanation.