Remote AI Benchmark Engineer & Multilingual Specialist

Lilt; Town of Belgium (WI); Hybrid; USD 90,000 - 150,000; Part time

Benefits offered by this job

Remote freelance

Flexible schedule

Competitive rates

Job summary

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmark tasks. You will create high-signal tasks in your native language, ensuring assets remain in the target language to measure multilingual handling without English translation crutches.

This is a remote, freelance opportunity with flexible scheduling. You’ll work on high-quality evaluation suites and participate in a rigorous QA process across creation, reviews, calibration, and

Qualifications

• 5+ years of industry experience in software engineering.
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
• Native or near-native fluency with deep understanding of grammar, register and phrasing rules. High English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
• Deep technical understanding of multilingual text processing pitfalls, incl. encoding/decoding robustness, Unicode normalization, locale conventions, and RTL handling.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using native-language datasets and files.
• Prompting & Translation: identify failure points in the native language.
• Implementation & Verification: support robust solutions and deterministic verifier scripts.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in 4-layer human and automated checks to ensure fairness and benchmark integrity.

Skills

Python

Shell scripting

Data processing

CLI workflows

Multilingual text processing

Education

Higher education degree in CS

Tools

Coding agents

Job description

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmark tasks. You will create high-signal tasks in your native language, ensuring assets remain in the target language to measure multilingual handling without English translation crutches.

This is a remote, freelance opportunity with flexible scheduling. You’ll work on high-quality evaluation suites and participate in a rigorous QA process across creation, reviews, calibration, and