Remote AI Benchmark Engineer — Multilingual Native Specialist

LILT (Production); United States; Remote; USD 70,000 - 120,000; Part time

Job summary

LILT (Production) is seeking experienced software engineers to design, build, and validate multilingual benchmarks for evaluating language models in native languages. You will create high-signal tasks, maintain assets in the target language, and test prompts without English translation crutches.

The role is remote and freelance, focusing on task engineering, dataset creation, and robust verifier scripts. Collaboration spans across a global linguistics and AI team, with flexible scheduling.

Qualifications

• 1+ years of industry experience in software or prompt engineering.
• Native or near-native fluency in the target language with high English proficiency.
• Strong proficiency in Python, shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows and coding agents.
• Deep understanding of multilingual text processing pitfalls.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build language-native task environments.
• Prompting & Translation: identify failure points in native language.
• Implementation & Verification: develop reference implementations and verifier scripts.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in 4-layer quality control and automated checks.

Skills

Python

Shell scripting

Data processing

Terminal/CLI workflows

Job description

LILT (Production) is seeking experienced software engineers to design, build, and validate multilingual benchmarks for evaluating language models in native languages. You will create high-signal tasks, maintain assets in the target language, and test prompts without English translation crutches.

The role is remote and freelance, focusing on task engineering, dataset creation, and robust verifier scripts. Collaboration spans across a global linguistics and AI team, with flexible scheduling.