AI Benchmark Engineer | Native Language Specialist - French (Belgium) - Remote

Lilt, Inc.

Belgique

Remote

EUR 83,000 - 165,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote work
Flexible schedule

Job summary

LILT (Production) seeks experienced software engineers to design, build, and validate rigorous multilingual benchmarks for testing multilingual robustness in AI systems. This remote, freelance role focuses on creating native-language tasks and robust verification tooling to push the limits of multilingual software challenges.

You will develop high-signal tasks and assets in the target languages, ensure they remain in the native language, and participate in a strict four-layer quality-control

Qualifications

  • 1+ years of industry experience in software engineering.
  • Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
  • Native or near-native fluency, with a deep understanding of its grammar.

Responsibilities

  • Task Engineering: Evaluating Coding Agents.
  • Asset Creation: Build realistic task environments using datasets and files in your native language. Crucially, these assets must remain in the target language to genuinely measure multilingual handling.
  • Prompting & Translation: finding failure points where AI does not work, in your native language.
  • Implementation & Verification: Support the development of robust solutions (reference implementations) and write highly reliable, deterministic verifier scripts (using rubric-based judging only when strictly necessary).
  • Calibration & Execution: Analyze execution logs and calibrate task difficulty (Easy to Very Hard) using standard Terminal-Bench run configurations against various model tiers (Haiku, Sonnet, Opus).
  • Quality Assurance: Participate in a rigorous, 4-layer human quality control process (creation, human review, calibration review, and audit) alongside automated LLM-based checks to ensure fairness, grammatical accuracy, and benchmark integrity.

Skills

Software engineering experience
Native language fluency

Education

Top-tier engineering university degree

Job description

About The Opportunity

We are building a rigorous, verifiable evaluation suite of Terminal-Bench tasks designed to test the limits of large language models on multilingual software challenges. Our goal is to measure multilingual robustness across prompt language effects, non-English data processing, and complex locale/encoding edge cases in terminal workflows. We are seeking experienced native-speaking software engineers to design, build, and validate these benchmarks. You will create high-signal, high-quality tasks that genuinely test a model's ability to handle multilingual environments without relying on English translation crutches. Note this is a remote, freelance opportunity

What You’ll Deliver
  • Task Engineering: Evaluating Coding Agents.
  • Asset Creation: Build realistic task environments using datasets and files in your native language. Crucially, these assets must remain in the target language to genuinely measure multilingual handling.
  • Prompting & Translation: finding failure points where AI does not work, in your native language Implementation & Verification: Support the development of robust solutions (reference implementations) and write highly reliable, deterministic verifier scripts (using rubric-based judging only when strictly necessary).
  • Calibration & Execution: Analyze execution logs and calibrate task difficulty (Easy to Very Hard) using standard Terminal-Bench run configurations against various model tiers (Haiku, Sonnet, Opus).
  • Quality Assurance: Participate in a rigorous, 4-layer human quality control process (creation, human review, calibration review, and audit) alongside automated LLM-based checks to ensure fairness, grammatical accuracy, and benchmark integrity.
Qualifications
  • Experience: 1+ years of industry experience in software engineering.
  • Background: Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
  • Language: Native or near-native fluency, with a deep understanding of its gramma
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Multilingual AI Benchmark Engineer (French) — Remote
Multilingual AI Benchmark Engineer (French) — Remote

Lilt, Inc. • Belgium

Remote
EUR 83,000 - 165,000
Remote work
Flexible schedule
AI Benchmark Engineer | Native Language Specialist - French (Belgium) - Remote
AI Benchmark Engineer | Native Language Specialist - French (Belgium) - Remote

Lilt, Inc. • Belgium

Remote
USD 90,000 - 150,000
Remote freelance
Flexible schedule
Competitive rates
British English Voice Expert
British English Voice Expert

Jobgether • Belgium

Remote
EUR 49,000 - 80,000
Fully remote
Competitive pay $40–$65/hr
12-month contract
+2
AI Engineer/Developer
AI Engineer/Developer

MEERAB Group • Brussel

On-site
EUR 70,000 - 110,000
French Audio Transcript Editor (AI Training)
French Audio Transcript Editor (AI Training)

Alignerr Corp. • Brussel Hoofdstad

Remote
EUR 21,000 - 41,000
Dutch Language Expert
Dutch Language Expert

SME Careers • Belgium

Remote
EUR 41,000 - 62,000
NLP Engineer
NLP Engineer

Jobgether • Belgium

Remote
EUR 80,000 - 120,000
Fully Remote Work
Competitive Salary
28 days vacation
+1
AI/ML Engineer (Brussels)
AI/ML Engineer (Brussels)

Bridge351 • Evere

Hybrid
EUR 65,000 - 100,000
Health Insurance
Life Insurance
Tech Visa Support
French Workspace Evaluator – Gmail & Google AI Review
French Workspace Evaluator – Gmail & Google AI Review

Alignerr Corp. • Brussel

Remote
EUR 34,000 - 48,000
Remote work
Flexible hours
AI Speech Tester (French native speaker - Belgium regional variant)
AI Speech Tester (French native speaker - Belgium regional variant)

TSMG Holding • Luik

On-site
EUR 27,552 - 41,328