Remote Urdu AI Evaluation & QA Analyst

Mercor; Dubai; On-site; AED 76,000 - 101,000; Part time

Job summary

Mercor seeks a Generalist to support English and Urdu language evaluation tasks for AI research projects. You will conduct fact-checking, generate evaluation data, assess reasoning, and ensure model outputs align with guidelines.

This remote contract role requires independent work and attention to detail. Applicants should hold a Bachelor's degree, be native Urdu speakers with strong English writing skills, and have experience with large language models and data annotation concepts.

Qualifications

• Bachelor's degree required.
• Native Urdu speaker.
• Significant experience with large language models (LLMs).
• Excellent writing skills in English.
• Strong attention to detail with analytical thinking.

Responsibilities

• Conduct fact-checking using trusted public sources and external tools.
• Generate high-quality evaluation data by identifying strengths and areas for improvement and factual inaccuracies.
• Assess reasoning quality, clarity, tone, and completeness of responses.
• Ensure model responses align with expected conversational behavior and system guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Skills

Urdu (native)

English writing

Fact-checking

LLM familiarity

Structured analytical thinking

Education

Bachelor's degree

Job description

Mercor seeks a Generalist to support English and Urdu language evaluation tasks for AI research projects. You will conduct fact-checking, generate evaluation data, assess reasoning, and ensure model outputs align with guidelines.

This remote contract role requires independent work and attention to detail. Applicants should hold a Bachelor's degree, be native Urdu speakers with strong English writing skills, and have experience with large language models and data annotation concepts.