Language Model Evaluator - Fully Remote | Upto $20/hr Part-time

mercor; India; Remote; INR 2.002.000 - 2.670.000; Teilzeit

Zusammenfassung

Mercor is seeking a Generalist - English & Bengali for a remote contract role. You will conduct fact-checking, generate high-quality evaluation data, and assess responses to improve AI models.

Ideal candidates are Bengali-native, have strong English writing skills, substantial experience with large language models (LLMs), and a background in analytical fields. You’ll work asynchronously to meet deadlines while maintaining rigor and quality.

Qualifikationen

• Bachelor’s degree required.
• Bengali native speaker with excellent English writing.
• Experience with large language models (LLMs) and data annotation is preferred.
• Strong attention to detail and structured analytical thinking.

Aufgaben

• Conduct fact-checking using trusted public sources and external tools.
• Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
• Assess reasoning quality, clarity, tone, and completeness of responses.
• Ensure model responses align with expected conversational behavior and system guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Kenntnisse

Bengali native speaker

LLM experience

English writing

Attention to detail

Analytical thinking

RLHF experience

Content writing/editing

Quality judgments

Ausbildung

Bachelor's degree

Jobbeschreibung

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist - English & Bengali

Type: Contract

Compensation: $15–$20/hour

Location: Remote

Responsibilities

• Conduct fact-checking using trusted public sources and external tools .
• Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
• Assess reasoning quality, clarity, tone, and completeness of responses.
• Ensure model responses align with expected conversational behavior and system guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

• Must-Have
• Bachelor's degree
• Native speaker in Bengali
• Significant experience using large language models (LLMs)
• Excellent writing skills in English
• Strong attention to detail
• Background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
• Preferred
• Prior experience with RLHF, model evaluation, or data annotation work
• Experience writing or editing high-quality written content
• Experience comparing multiple outputs and making fine-grained qualitative judgments