Korean Language AI Evaluator & Prompt Architect

Agency; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Agency seeks a rigorous linguistics researcher to evaluate language models in Korean. You will engage models, assess accuracy, document failure modes, and propose metrics and prompt improvements.

You will also help refine evaluation protocols and ensure robust linguistic coverage across dialects. Ideal candidates have deep knowledge of Korean linguistics, can provide audio/video/text inputs, and maintain a secure setup.

Qualifications

• Expertise in Korean grammar, syntax, morphology, phonetics, semantics, and pragmatics.
• Ability to provide clear, metacognitive communication.
• Must be 18 years or older.
• Ability to provide audio, video, image, and written inputs.
• Access to a secure computer and high-speed internet.

Responsibilities

• Converse with language models on various language scenarios.
• Verify factual accuracy and logical soundness of model outputs.
• Capture reproducible error traces and document failure modes.
• Suggest improvements to prompt engineering and evaluation metrics.
• Challenge models on honorifics, syntax, morphology, and regional dialects.

Skills

Korean grammar

Metacognitive communication

Language model evaluation

Education

Master's or PhD in Korean linguistics or related field

Job description

Agency seeks a rigorous linguistics researcher to evaluate language models in Korean. You will engage models, assess accuracy, document failure modes, and propose metrics and prompt improvements.

You will also help refine evaluation protocols and ensure robust linguistic coverage across dialects. Ideal candidates have deep knowledge of Korean linguistics, can provide audio/video/text inputs, and maintain a secure setup.