AI Personalization QA Specialist

Hire Feed; United States; Remote; USD 60,000 - 90,000; Full time

Job summary

Hire Feed is seeking an AI Quality Analyst to evaluate a new personalization feature for an AI model. The role involves assessing how well it integrates information from past conversations, email, search, and activity to deliver relevant responses.

You will craft creative prompts and analytically assess AI outputs, focusing on grounding, integration, and helpfulness, to improve personalization performance. This is a full-time position based in the United States.

Qualifications

• Excellent written and spoken English required.
• Experience designing prompts or test cases for AI models.
• Ability to analyze nuanced and ambiguous AI responses with rigor.
• Familiarity with personalization concepts and evaluation metrics in AI systems.
• Strong attention to detail to spot subtle discrepancies in outputs.
• Willingness to use your personal Google account and enable data sources for assessment.
• Full-time availability in your local time zone to support a global operations team.

Responsibilities

• Evaluate quality of personalized responses from an AI model using your own experiences as context.
• Design creative, multi-turn prompts to test integration of personal data sources.
• Assess model responses for grounding, integration, and usefulness; flag issues.
• Review side-by-side comparisons to detect differences in relevance and accuracy.
• Provide detailed feedback to improve personalization performance.

Skills

English proficiency

Prompt engineering

Analytical reasoning

Personalization concepts

Attention to detail

Personal data sources

Local time zone availability

Job description

Hire Feed is seeking an AI Quality Analyst to evaluate a new personalization feature for an AI model. The role involves assessing how well it integrates information from past conversations, email, search, and activity to deliver relevant responses.

You will craft creative prompts and analytically assess AI outputs, focusing on grounding, integration, and helpfulness, to improve personalization performance. This is a full-time position based in the United States.