Remote AI Personalization Quality Analyst

Turing; Seattle (WA); On-site; USD 55,000 - 83,000; Full time

Job summary

Turing, a leading AI research partner, seeks an AI Quality Analyst to evaluate a personalization feature for Gemini, using your own conversations and online activity to test grounding, integration, and helpfulness.

You will design multi-turn prompts, assess model responses, provide clear rationales with references to turn numbers, and maintain data hygiene by deleting evaluation conversations.

Qualifications

• BS/BA degree or equivalent in a relevant field.
• Experience in data annotation, AI quality evaluation, content moderation, or a related role preferred.

Responsibilities

• Design and execute multi-turn prompts that require the AI to utilize personal information and experiences.
• Evaluate model responses for personalization intent and correctness.
• Analyze responses for grounding issues and ensure claims are supported by evidence.
• Assess integration quality to weave personal data naturally without over-narration.
• Rigorously stack-rank two model responses (SxS) for overall usefulness and clarity.
• Write clear, defensible rationales referencing turn numbers and evidence.
• Extract and verify "Debug Info" to confirm proper data source usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Skills

English proficiency

Personal account usage

Schedule flexibility

Analytical thinking

Prompt engineering

Evaluation acumen

Attention to detail

Written communication

Feedback

Communication

Independence

Technical setup

Education

BS/BA degree or equivalent

Job description

Turing, a leading AI research partner, seeks an AI Quality Analyst to evaluate a personalization feature for Gemini, using your own conversations and online activity to test grounding, integration, and helpfulness.

You will design multi-turn prompts, assess model responses, provide clear rationales with references to turn numbers, and maintain data hygiene by deleting evaluation conversations.