AI Data Reviewer | $20/hr Remote

Crossing Hurdles; United States; On-site; USD 21,000 - 28,000; Full time

Job summary

Crossing Hurdles is seeking an LLM - AI Quality Analyst (Personalization) to work remotely on a four-month contractor assignment. The role focuses on evaluating and improving personalization in AI responses, with strong emphasis on English language prompts and outputs.

You will design and test multi-turn prompts, assess model performance, provide structured feedback, and ensure data-handling procedures are followed.

Qualifications

• Strong English proficiency in reading and writing.
• High attention to detail and analytical thinking.
• Ability to evaluate nuanced AI responses.
• Experience creating and evaluating multi-turn prompts.
• Reliable remote work setup.

Responsibilities

• Design and evaluate multi-turn prompts with personalization.
• Assess AI responses for personalization and usefulness.
• Compare model outputs side-by-side for quality.
• Identify inaccurate personalization and unsupported inferences.
• Provide written rationales and feedback on performance.
• Follow data-handling procedures and review information use.

Skills

English proficiency

Analytical thinking

Attention to detail

Prompt evaluation

Independent work

Remote work

Written communication

Job description

Position: LLM - AI Quality Analyst (Personalization) - English

Type: Short-Term Contractor Assignment (4 months)

Compensation: $15 - $20 per hour

Location: Remote

Commitment: Up to 40 hours per week with PST overlap

Role Responsibilities

• Design and evaluate multi-turn prompts using personal context and experiences.
• Assess AI responses for personalization, grounding, integration, and helpfulness.
• Compare model responses side-by-side and evaluate their overall quality.
• Identify inaccurate personalization, unsupported inferences, and other response issues.
• Provide clear written rationales and detailed feedback on model performance.
• Review the use of relevant information in responses and follow project data-handling procedures.

Requirements

• Strong English proficiency, including reading and writing.
• Strong analytical thinking and attention to detail.
• Ability to evaluate nuanced and ambiguous AI responses.
• Ability to create and assess multi-turn conversational prompts.
• Strong written communication skills and ability to provide structured feedback.
• Ability to work independently in a remote environment.
• Reliable computer and internet connection.
• Willingness to use a personal Google account and required data sources for the evaluation process.
• Full-time availability in the local time zone for applicable locations.