Basque Language Scientist & Prompt-Evaluation Specialist

Agency; United States; Remote; USD 70,000 - 120,000; Full time

Job summary

Agency is seeking a specialist to collaborate on Basque language model testing and evaluation. You will converse with language models across Basque scenarios, verify factual accuracy, and document failure modes to inform improvements.

The role requires deep expertise in Basque linguistics and the ability to articulate metacognitive reasoning, showing your work. A secure computing setup and fast internet are essential.

Qualifications

• Expertise in Basque grammar, syntax, morphology, spelling, phonology, and pragmatics.
• Ability to provide clear, metacognitive communication and show your work.
• Access to a secure computer and high-speed internet.

Responsibilities

• Converse with language models on various Basque language scenarios.
• Verify factual accuracy and logical soundness of model outputs.
• Capture reproducible error traces and document failure modes.
• Suggest improvements to prompt engineering and evaluation metrics.
• Challenge models on grammar, syntax, morphology, and dialectal variation.

Skills

Basque grammar

Basque syntax

Basque morphology

Basque spelling

Basque phonology

Basque pragmatics

Metacognitive communication

Secure computing environment

Education

Master's or PhD in Basque language/linguistics

Job description

Agency is seeking a specialist to collaborate on Basque language model testing and evaluation. You will converse with language models across Basque scenarios, verify factual accuracy, and document failure modes to inform improvements.

The role requires deep expertise in Basque linguistics and the ability to articulate metacognitive reasoning, showing your work. A secure computing setup and fast internet are essential.