LLM AI Quality Analysts

Donato Technologies Inc; Moraga (CA); Remote; USD 17,000 - 25,000; Part time

Benefits offered by this job

Remote work

Job summary

Donato Technologies Inc. is seeking an LLM / AI Quality Analyst (Personalization) for a 4-month remote contract. The role focuses on evaluating personalized AI responses, creating prompts, and assessing grounding and helpfulness to improve model behavior.

Requirements include Bachelor’s or Master’s in IT or related fields, strong English skills, and the ability to work remotely with a reliable setup. A flexible 30–40 hour workweek and immediate start are expected for this contractor position.

Qualifications

• Strong English reading and writing skills.
• Strong analytical and attention-to-detail skills.
• Experience with AI evaluation, data annotation, content moderation, or related work preferred.
• Ability to work independently in a remote environment.
• Laptop/desktop with reliable internet connection.
• Willingness to use a personal Google account and enable personal data sources for the assessment.

Responsibilities

• Evaluate personalized AI/Gemini responses.
• Create and test multi-turn prompts.
• Assess AI responses for grounding, integration, and helpfulness.
• Compare model responses side-by-side.
• Identify incorrect personalization, poor inferences, and hallucinations.
• Provide detailed feedback and rationales.

Skills

English proficiency

Analytical skills

AI evaluation

Remote work

Tech setup

Google account usage

Education

Bachelor’s/Master’s in IT or related field

Tools

Google account

Job description

IMMEDIATE HIRING | LLM / AI QUALITY ANALYST

We are hiring for LLM – AI Quality Analysts (Personalization) – English for a 4-month remote contract opportunity. Rate: $15/hr USD Start Date: Immediately Availability: 30–40 hours/week PST Overlap: 4 hours required Engagement: Contractor

Education: Bachelor’s or Master’s degree in IT or any IT-related field

Approved States: Alabama, Arizona, Colorado, Florida, Idaho, Iowa, Kentucky, Michigan, Minnesota, Mississippi, Missouri, Montana, New York, North Carolina, North Dakota, Oklahoma, Pennsylvania, South Carolina, South Dakota, Virginia, Wisconsin, Wyoming

Requirements

• Strong English reading and writing skills
• Strong analytical and attention-to-detail skills
• AI evaluation, data annotation, content moderation, or related experience preferred
• Ability to work independently in a remote environment
• Laptop/Desktop with reliable internet
• Willingness to use a primary personal Google account and enable personal data sources for the assessment

What You’ll Do

• Evaluate personalized AI/Gemini responses
• Create and test multi-turn prompts
• Assess AI responses for Grounding, Integration, and Helpfulness
• Compare model responses side-by-side
• Identify incorrect personalization, poor inferences, and hallucinations
• Provide detailed feedback and rationales

This is a remote position.