LLM Quality Evaluator — Shape AI Conversations

Mercor; À distance; EUR 40 000 - 60 000; Plein temps

Résumé du poste

A prominent AI talent connection firm is seeking an AI Model Evaluation Specialist to evaluate LLM-generated responses and ensure quality. The ideal candidate will possess a Bachelor's degree, be a native German speaker, and have extensive experience with LLMs. Strong writing skills and attention to detail are essential. This role offers flexible working arrangements with competitive hourly compensation based on your expertise.

Qualifications

• Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in German.
• Experience writing or editing high-quality content is a plus.
• Excellent college-level mathematics skills.

Responsabilités

• Evaluate LLM-generated responses for effectiveness.
• Conduct fact-checking using reliable sources.
• Generate annotated feedback for LLM responses.
• Ensure adherence to conversational expectations.
• Apply consistent annotations using detailed guidelines.

Connaissances

Fluent in German

Experience with LLMs

Strong writing skills

Attention to detail

Analytical thinking

Formation

Bachelor’s degree

Description du poste

A prominent AI talent connection firm is seeking an AI Model Evaluation Specialist to evaluate LLM-generated responses and ensure quality. The ideal candidate will possess a Bachelor's degree, be a native German speaker, and have extensive experience with LLMs. Strong writing skills and attention to detail are essential. This role offers flexible working arrangements with competitive hourly compensation based on your expertise.