Remote AI Quality Analyst: Personalization & Evaluation

Turing; New York (NY); Remote; USD 34,000 - 83,000; Part time

Benefits offered by this job

Contractor engagement (3 months)

Flexible daily schedule

Job summary

Turing is seeking an AI Quality Analyst to evaluate a personalization feature for Gemini. You will design prompts based on your own experiences and assess how well the model uses information from conversations, Gmail, Google Search, and YouTube activity to improve responses.

You will rank competing model outputs, justify your choices with precise turn references, and verify that chat summaries and data sources were properly used.

Qualifications

• BS/BA degree or equivalent in a relevant field.
• Experience in data annotation, AI quality evaluation, content moderation, or related role preferred.

Responsibilities

• Design and execute multi-turn prompts (1–5 turns) requiring personal data usage.
• Evaluate model responses for personalization effectiveness and grounding.
• Analyze responses for grounding, integration, and helpfulness.
• Rank two model responses side-by-side (SxS) and provide defensible rationales referencing turns.
• Extract and verify Debug Info from the model to confirm data sources usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Skills

English proficiency

Time zone flexibility

Analytical thinking

Creative prompt engineering

Evaluation acumen

Attention to detail

Written communication

Feedback and annotations

Independence

Technical setup

Education

BS/BA degree or equivalent

Experience in data annotation/AI quality evaluation

Job description

Turing is seeking an AI Quality Analyst to evaluate a personalization feature for Gemini. You will design prompts based on your own experiences and assess how well the model uses information from conversations, Gmail, Google Search, and YouTube activity to improve responses.

You will rank competing model outputs, justify your choices with precise turn references, and verify that chat summaries and data sources were properly used.