Content Writer

Innodata Inc.; United Kingdom; Hybrid; ZAR 60,000 - 120,000; Part time

Job summary

Innodata Inc. is seeking a part-time LLM Evaluation Specialist to enhance multilingual datasets for large language models. The role requires native proficiency in English, cultural insights, and experience in LLM evaluation, along with responsibilities such as rubric design and prompt creation. This opportunity combines writing, editing, and critical evaluation within the growing fields of IT services and consulting.

Qualifications

• Native proficiency in English and familiarity with cultural norms.
• Experience in LLM evaluation, content moderation, or linguistic QA preferred.
• Strong attention to detail with ability to identify subtle language issues.

Responsibilities

• Update rubric definitions for cultural relevance.
• Review translations and write thoughtful prompts.
• Document problematic outputs and annotate rubric violations.

Skills

Attention to Detail

Cultural Awareness

Linguistic QA

Prompt Engineering

Education

Master’s degree in relevant stream

Tools

Gemini

ChatGPT

Job description

Direct message the job poster from Innodata Inc.

Head HR - CoE | Talent & Global Sourcing at Innodata Inc.

Job Title: LLM Evaluation Specialist – Cultural and Linguistic Alignment (English)

We are looking for linguistically and culturally aware professionals to support the evaluation and enhancement of multilingual prompt-response datasets for large language models (LLMs). This role involves rubric design, evaluation of translations and model outputs, prompt creation, and red teaming focused on identifying and surfacing cultural nuances and biases in LLM behavior.

Key Responsibilities:

Rubric Definition & Prompt Evaluation

• Update rubric definitions with region/language-specific examples to ensure cultural and linguistic relevance.
• Identify the need for additional rubrics tailored to specific languages or regional contexts.
• Review prompts translated from English into the target language and revise where translations appear unnatural or inaccurate.
• Writing of thoughtful prompts which can test the cultural awareness of LLM models.
• Rate prompt-response pairs using a standardized evaluation template based on rubrics and provide detailed justifications to base the findings.
• Document problematic outputs and annotate them with clear explanations of rubric violations or cultural insensitivities.

Required Qualifications:

• Native proficiency in the target language (English) and deep familiarity with cultural norms in the corresponding region.
• Experience in LLM evaluation, content moderation, or linguistic QA preferred.
• Strong attention to detail with the ability to identify subtle issues in language use, tone, and cultural references.
• Comfortable working in spreadsheets and evaluation templates.
• Master’s degree in relevant stream.

Preferred Qualifications:

• Prior experience with prompt engineering or LLM testing.
• Familiarity with tools such as Gemini, ChatGPT or similar LLM platforms.
• Ability to clearly articulate reasoning behind rubric ratings or prompt edits.

Required Engagement: 25 to 30 hour per week.

Seniority level

• Seniority level

  Associate

Employment type

• Employment type

  Part-time

Job function

• Job function

  Writing/Editing and Art/Creative

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Innodata Inc. by 2x

Get notified about new Content Writer jobs in United Kingdom.

London, England, United Kingdom 4 days ago

Southampton, England, United Kingdom 1 week ago

Glasgow, Scotland, United Kingdom 1 week ago

London, England, United Kingdom 2 months ago

Coventry, England, United Kingdom 1 week ago

United Kingdom ZAR60,000 - ZAR120,000 1 week ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.