Quality Analyst | $11/hr Remote

Crossing Hurdles; India; On-site; INR 334,800 - 558,000; Part time

Job summary

A leading technology firm is seeking an AI Quality Analyst to evaluate personalized AI responses and ensure data hygiene. The role requires strong English fluency and experience in data annotation or AI quality evaluation. Responsibilities include designing conversational prompts, assessing AI responses, and providing structured feedback. This is a remote, short-term contract that demands a reliable internet connection and the ability to work independently.

Qualifications

• Fluent in English, both reading and writing.
• Proven experience in data annotation or AI quality evaluation.
• Ability to evaluate nuanced and ambiguous responses.

Responsibilities

• Design multi-turn conversational prompts based on personal context.
• Evaluate personalized AI responses for relevance and helpfulness.
• Assess the use of personal data in model outputs.

Skills

English fluency (reading and writing)

Strong analytical thinking

Attention to detail

Data annotation experience

AI quality evaluation

Content moderation experience

Strong written communication

Education

BS/BA degree or equivalent experience in a relevant analytical field

Job description

Position: LLM – AI Quality Analyst (Personalization) – English

Type: Short-Term Contract

Location: Remote

Commitment: 30–40 hours/week

Engagement Length: 2 months

Start Date: Immediate

Responsibilities

• Design multi-turn conversational prompts based on personal context
• Evaluate personalized AI responses for relevance, grounding, and helpfulness
• Assess correct and incorrect use of personal data in model outputs
• Perform side‑by‑side (SxS) evaluation and ranking of AI responses
• Identify grounding errors, poor inferences, and forced personalization
• Write clear, structured rationales referencing specific conversation turns
• Extract and verify model debug information and data source usage
• Maintain strict data hygiene by deleting evaluation conversations

Qualifications

• English fluency (reading and writing)
• Strong experience in data annotation, AI quality evaluation, content moderation, or related roles
• Strong analytical thinking and attention to detail
• Ability to evaluate nuanced and ambiguous AI responses
• Comfortable using a primary personal Google account with enabled data sources
• BS/BA degree or equivalent experience in a relevant analytical field
• Strong written communication and structured feedback skills
• Self‑motivated and able to work independently in a remote setting
• Reliable desktop/laptop with stable internet connection