AI Personalization QA Analyst

Turing; Seattle (WA); Remote; USD 34,000 - 69,000; Part time

Job summary

Turing, based in San Francisco, seeks an AI Quality Analyst to evaluate a new personalization feature for Gemini. You will design prompts from your own experiences and assess how well the model uses past conversations and online activity to tailor responses.

You will rank model outputs, document rationales with turn-number references, and help improve grounding, integration, and usefulness while maintaining data hygiene and clear feedback.

Qualifications

• BS/BA degree or equivalent in a relevant field.
• Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred.

Responsibilities

• Design and run multi-turn prompts testing personalization using your own experiences.
• Evaluate model responses for alignment with user intent and whether personalization was applied.
• Analyze Grounding to ensure claims are evidenced and not hallucinations.
• Assess Integration quality to weave personal data naturally without over-narrating.
• Rigorously stack-rank two model responses (SxS) for overall usefulness and clarity.
• Write clear, defensible rationales referencing specific turn numbers.
• Extract and verify Debug Info from the model to confirm proper data sources usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Skills

Analytical Thinking

Prompt Engineering

Evaluation Acumen

Attention to Detail

Written Communication

Feedback

Communication

Independence

Technical Setup

Education

BS/BA degree or equivalent

Job description

Turing, based in San Francisco, seeks an AI Quality Analyst to evaluate a new personalization feature for Gemini. You will design prompts from your own experiences and assess how well the model uses past conversations and online activity to tailor responses.

You will rank model outputs, document rationales with turn-number references, and help improve grounding, integration, and usefulness while maintaining data hygiene and clear feedback.