AI Personalization Quality Analyst & Prompt Designer

Turing; New York (NY); On-site; USD 34,000 - 62,000; Part time

Benefits offered by this job

Contractor engagement

4 hours overlap with PST

Remote work

Job summary

Turing is seeking an AI Quality Analyst to evaluate a personalization feature for Gemini. You will design multi-turn prompts based on personal context and assess grounding, integration, and helpfulness of model responses.

This remote contractor role requires English proficiency, meticulous attention to detail, and the ability to provide structured rationales with turn references. Commitment of 4 hours daily, up to 40 hours weekly, with PST overlap.

Qualifications

• BS/BA degree or equivalent experience in a relevant field.
• Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred.

Responsibilities

• Designing and executing multi-turn conversational prompts (typically 1-5 turns) that require the AI to utilize your personal information and experiences.
• Evaluating model responses based on your intent from the starting prompt, checking if the personalization was appropriately applied.
• Analyzing responses for Grounding issues, ensuring claims about you are supported by evidence and not flawed inferences or hallucinations.
• Assessing Integration quality to ensure personal data is woven naturally into the response without robotic 'overnarrating'.
• Rigorously evaluating and stack-ranking two model responses side-by-side (SxS) to determine which is overall more helpful, easy to use, and enjoyable.
• Writing clear, defensible rationales for your comparisons, explicitly referencing where issues or positive aspects occurred in the conversation.
• Extracting and verifying "Debug Info" from the model to confirm that chat summaries and data sources were properly utilized.
• Maintaining strict data hygiene by deleting evaluation conversations to prevent them from polluting your future chat history.

Skills

English Proficiency

Personal Account Usage

Schedule Flexibility

Exceptional Analytical Thinking

Creative Prompt Engineering

Strong Evaluation Acumen

Meticulous Attention to Detail

Excellent Written Communication

Feedback

Communication

Independence

Technical Setup

Education

BS/BA degree or equivalent in a relevant field

Job description

Turing is seeking an AI Quality Analyst to evaluate a personalization feature for Gemini. You will design multi-turn prompts based on personal context and assess grounding, integration, and helpfulness of model responses.

This remote contractor role requires English proficiency, meticulous attention to detail, and the ability to provide structured rationales with turn references. Commitment of 4 hours daily, up to 40 hours weekly, with PST overlap.