Remote AI Benchmark Engineer - Native French Specialist

LILT; United States; Remote; USD 83,000 - 165,000; Full time

Job summary

LILT is seeking experienced native-speaking software engineers for a remote, freelance opportunity to design, build, and validate benchmarks that test multilingual robustness in language models. You will create high-signal tasks in your native language, build realistic task environments, and write reliable verifier scripts while participating in a rigorous quality-control process.

You’ll leverage Python, shell scripting, and data processing skills, with a strong emphasis on multilingual text

Qualifications

• 5+ years of industry experience in software engineering.
• Native or near-native fluency with high English proficiency.
• Strong Python, shell scripting, and data processing skills.
• Extensive experience with Terminal/CLI-based development workflows and coding agents.
• Deep technical understanding of multilingual text processing pitfalls.

Responsibilities

• Design, build, and validate benchmarks for multilingual model evaluation.
• Create high-signal, high-quality tasks in native language.
• Develop assets and verification scripts; provide reference implementations.
• Calibrate task difficulty across model tiers using Terminal-Bench configurations.
• Participate in a four-layer quality control process with automated checks.

Skills

Software engineering

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

Education

Top-tier engineering universities

Job description

LILT is seeking experienced native-speaking software engineers for a remote, freelance opportunity to design, build, and validate benchmarks that test multilingual robustness in language models. You will create high-signal tasks in your native language, build realistic task environments, and write reliable verifier scripts while participating in a rigorous quality-control process.

You’ll leverage Python, shell scripting, and data processing skills, with a strong emphasis on multilingual text