Remote Multilingual AI Benchmark Engineer (Freelance)

LILT (Production); Netherlands; Remote; EUR 70,000 - 100,000; Part time

Benefits offered by this job

Flexible schedule

Prompt payments

Impactful projects

Global community

Portfolio growth

Fun work

Job summary

LILT (Production) is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks for Terminal-Bench tasks. You will create high-signal tasks in your native language, preserving native text to accurately assess multilingual handling, and contribute to a rigorous four-layer quality process.

As a remote contractor, you’ll work flexibly across projects, set your schedule, and receive competitive rates.

Qualifications

• 5+ years of software engineering experience in industry.
• Native or near-native fluency with deep grammar and phrasing knowledge.
• High proficiency in English and strong technical stack skills.

Responsibilities

• Design and build multilingual benchmarks for Terminal-Bench tasks.
• Create high-signal tasks in the native language without English translation crutches.
• Develop verification scripts and reference implementations.
• Calibrate task difficulty across model tiers (Easy to Very Hard).
• Participate in a four-layer human quality control process.
• Ensure fairness, grammar accuracy, and benchmark integrity.

Skills

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

5+ years experience

English proficiency

Education

Top-tier engineering universities

Tools

Python

Shell scripting

Data processing

Job description

LILT (Production) is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks for Terminal-Bench tasks. You will create high-signal tasks in your native language, preserving native text to accurately assess multilingual handling, and contribute to a rigorous four-layer quality process.

As a remote contractor, you’ll work flexibly across projects, set your schedule, and receive competitive rates.