Dutch AI Benchmark Engineer & Multilingual Evaluator

LILT AI

Amsterdam

Remote

EUR 83,000 - 165,000

Part time

11 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Flexible schedule
Competitive pay
Remote work possible

Job summary

LILT is seeking native-speaking software engineers to design, build, and validate a rigorous multilingual evaluation suite for Terminal-Bench tasks. You will craft high-signal tasks in the native language and ensure assets remain in the target language, testing multilingual robustness without English translation crutches.

The role emphasizes Python, shell scripting, and data processing, with focus on encoding, locale handling, and RTL concerns.

Qualifications

  • 1+ years of industry experience in software engineering
  • Graduation from top-tier engineering universities
  • Native or near-native fluency in the target language with high English proficiency
  • Strong Python, shell scripting and data processing skills
  • Experience with Terminal/CLI-based development workflows and coding agents
  • Deep understanding of multilingual text processing pitfalls, including encoding/decoding and locale rules

Responsibilities

  • Design, build and validate benchmarks for multilingual software challenges
  • Create high-signal tasks in the native language and ensure assets remain in the target language
  • Develop prompts, translation checks, and deterministic verifier scripts
  • Analyze model outputs and calibration data across languages and locales
  • Collaborate in a 4-layer human quality control process (creation, review, calibration, audit)

Skills

Native language fluency
English proficiency
Python
Shell scripting
Data processing
Terminal/CLI workflows
Coding agents
Multilingual text processing

Education

Engineering degree from top-tier university

Tools

Python
Shell scripting
Data processing

Job description

LILT is seeking native-speaking software engineers to design, build, and validate a rigorous multilingual evaluation suite for Terminal-Bench tasks. You will craft high-signal tasks in the native language and ensure assets remain in the target language, testing multilingual robustness without English translation crutches.

The role emphasizes Python, shell scripting, and data processing, with focus on encoding, locale handling, and RTL concerns.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Benchmark Engineer | Native Language Specialist - Dutch - Remote
AI Benchmark Engineer | Native Language Specialist - Dutch - Remote

LILT AI • Amsterdam

Remote
EUR 83,000 - 165,000
Flexible schedule
Competitive pay
Remote work possible
Remote LLM Evaluation Engineer - AI Research & Benchmarks
Remote LLM Evaluation Engineer - AI Research & Benchmarks

Blue Lynx Employment BV • Netherlands

Remote
EUR 60,000 - 80,000
25 holidays per annum
Pension plan
Opportunity to work on cutting-edge AI initiatives
Remote Bilingual AI Evaluator (Dutch/German/French)
Remote Bilingual AI Evaluator (Dutch/German/French)

10x Team • Netherlands

Remote
EUR 34,000 - 55,000
Software Engineer, LLM Evaluation – English
Software Engineer, LLM Evaluation – English

Blue Lynx Employment BV • Netherlands

Remote
EUR 60,000 - 80,000
25 holidays per annum
Pension plan
Opportunity to work on cutting-edge AI initiatives
Dutch Language Quality Assessor & AI Speech Evaluator
Dutch Language Quality Assessor & AI Speech Evaluator

Crossing Hurdles • Netherlands

Remote
EUR 28,000 - 41,000
Dutch Language AI Trainer & LLM Evaluator
Dutch Language AI Trainer & LLM Evaluator

Agency • Netherlands

Remote
EUR 7,400 - 80,000
Translator (Dutch - English) | $65/hr Remote
Translator (Dutch - English) | $65/hr Remote

Crossing Hurdles • Netherlands

Remote
EUR 28,000 - 41,000
Dutch Quality Control Specialist - Project Lyra
Dutch Quality Control Specialist - Project Lyra

Welo Global • Arnhem

Hybrid
EUR 44,000 - 61,000
Fully remote
Dutch Science AI Trainer for LLMs (Remote)
Dutch Science AI Trainer for LLMs (Remote)

Invisible Agency • Netherlands

Remote
EUR 9,800 - 80,000
Dutch Prompt Evaluator — AI Language Tech in Essen
Dutch Prompt Evaluator — AI Language Tech in Essen

Cross Border Talents • Rotterdam

On-site
EUR 42,000 - 64,000
Competitive salary package
Relocation package
Welcome bonus
+1