Quality Analyst (Remote)

Hired; United States; Remote; GBP 36,000 - 52,000; Full time

Job summary

Hired is seeking an AI Quality Analyst (English) to evaluate a new personalization feature within an AI model on a full-time basis. You will assess how well the system integrates information from past conversations, emails, search and activity to deliver relevant responses.

In this role you’ll design prompts, analyze AI outputs for grounding and helpfulness, and provide detailed feedback to improve personalization performance.

Qualifications

• English proficiency for reading and writing at a high level.
• Experience designing tests or prompts for AI models.
• Ability to analytically evaluate nuanced AI responses.
• Familiarity with personalization concepts and evaluation metrics.
• Attention to detail to spot discrepancies in outputs.
• Willingness to use a personal Google account and enable data sources.
• Full-time availability in local UK time zone for global operations.

Responsibilities

• Evaluate quality of personalized responses from an AI model using personal experience as context.
• Design creative, multi-turn prompts to test integration of data sources.
• Assess model responses for grounding, integration and usefulness.
• Review side-by-side AI responses to detect differences in relevance and accuracy.
• Provide detailed feedback to improve personalization performance.

Skills

English proficiency

Prompt engineering

Analytical evaluation

Personalization concepts

Attention to detail

Data source assessment

Global operations availability

Job description

We are hiring for one of our clients, seeking an AI Quality Analyst - English to work on a full-time basis. You will evaluate a new personalization feature for an AI model by assessing how well it integrates information from past conversations, email, search, and activity to deliver relevant responses. This role requires a combination of creative prompt design and analytical evaluation of AI-generated content.

Key Responsibilities:

• Evaluate the quality of personalized responses from an AI model using your own personal experiences as context.
• Design creative, multi-turn prompts that test the model’s ability to integrate personal data sources accurately.
• Assess model responses for grounding, integration, and helpfulness, identifying incorrect personalization or forced connections.
• Review side-by-side comparisons of AI responses to detect subtle differences in relevance and accuracy.
• Provide detailed feedback on model behavior to improve personalization performance.

Required Skills & Qualifications:

• English proficiency with a high degree of competence in reading and writing.
• Experience in prompt engineering or designing test cases for AI models.
• Ability to analyze nuanced and ambiguous AI responses with analytical rigor.
• Familiarity with personalization concepts and evaluation metrics in AI systems.
• Attention to detail to identify subtle discrepancies in model outputs.
• Willingness to use your primary personal Google account and enable personal data sources for assessment.
• Full-time availability in your local time zone to support a global, 24-hour operations team.

More About the Opportunity:

This role offers a unique opportunity to work with a global leader in the Technology, Information and Internet industry, contributing to the development of advanced AI systems that deliver measurable business impact.

Equal Opportunity Employer:

We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history. Applications are reviewed solely on demonstrated technical ability and qualifications.