Remote AI Benchmark Engineer — French Language Specialist

LILT (Production); France; Hybride; EUR 74 000 - 129 000; Temps partiel

Résumé du poste

Lilt is seeking a native-speaking software engineer for a remote, freelance engagement to design, build, and validate multilingual benchmarks. You will craft high-signal tasks, datasets, and deterministic verifier scripts, using rubric-based judging only where necessary.

Responsibilities cover task engineering, asset creation in your native language, prompting and translation, implementation and verification, calibration of task difficulty, and rigorous quality assurance across a 4-layer review

Qualifications

• 5+ years of software engineering experience.
• Native or near-native fluency with deep understanding of grammar, register and phrasing rules.
• Strong proficiency in Python, shell scripting, and data processing.

Responsabilités

• Task Engineering: evaluate coding agents and build realistic task environments.
• Asset Creation: develop language-specific assets that stay in the target language.
• Prompting & Translation: identify failure points in native language.
• Implementation & Verification: create robust reference implementations and deterministic verifier scripts.
• Calibration & Execution: adjust task difficulty across model tiers with standard configurations.
• Quality Assurance: participate in a 4-layer human and automated QC process.

Connaissances

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual NLP

Unicode normalization

Locale handling

RTL handling

Outils

Verifier scripts

Description du poste

Lilt is seeking a native-speaking software engineer for a remote, freelance engagement to design, build, and validate multilingual benchmarks. You will craft high-signal tasks, datasets, and deterministic verifier scripts, using rubric-based judging only where necessary.

Responsibilities cover task engineering, asset creation in your native language, prompting and translation, implementation and verification, calibration of task difficulty, and rigorous quality assurance across a 4-layer review