Remote AI Benchmark Engineer — French Language Specialist

LILT (Production); France; Hybride; EUR 69 000 - 138 000; Plein temps

Résumé du poste

LILT is building a rigorous multilingual evaluation suite to test large language models on non-English data and locale-aware processing. We are seeking experienced native-speaking software engineers to design, build, and validate benchmarks that stay in the target language.

This remote freelance opportunity requires 5+ years in software engineering, deep Python and shell scripting skills, and a keen eye for multilingual text issues.

Qualifications

• 5+ years of industry experience in software engineering.
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
• Native or near-native fluency, with a deep understanding of its grammar, register, and phrasing rules. High English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
• Deep technical understanding of multilingual text processing pitfalls, including encoding/normalization, locale conventions, and safe string operations.

Responsabilités

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets and files in your native language.
• Prompting & Translation: identify failure points in your native language.
• Implementation & Verification: develop reference implementations and deterministic verifier scripts.
• Calibration & Execution: analyze logs and calibrate task difficulty across model tiers.
• Quality Assurance: participate in a 4-layer QA process to ensure fairness and accuracy.

Connaissances

Python

Shell scripting

Data processing

Terminal/CLI

Multilingual text

Formation

Engineering degree

Outils

Python

Shell scripting

Data processing tools

Description du poste

LILT is building a rigorous multilingual evaluation suite to test large language models on non-English data and locale-aware processing. We are seeking experienced native-speaking software engineers to design, build, and validate benchmarks that stay in the target language.

This remote freelance opportunity requires 5+ years in software engineering, deep Python and shell scripting skills, and a keen eye for multilingual text issues.