Remote Indonesian AI Benchmark Engineer

LILT (Production); United States; Remote; USD 83,000 - 165,000; Full time

Benefits offered by this job

Remote work

Flexible schedule

Competitive rates

Job summary

LILT is building a rigorous evaluation suite of Terminal-Bench tasks designed to test multilingual robustness across prompt language effects, non-English data processing, and locale/encoding edge cases in terminal workflows.

We seek experienced native-speaking software engineers to design, build, and validate these benchmarks. You will create high-signal tasks that test a model’s native-language handling without relying on English translation. This is a remote, freelance opportunity.

Qualifications

• 5+ years of industry experience in software engineering.
• Graduation from top-tier engineering universities.
• Native or near-native fluency with high English proficiency.
• Strong proficiency in Python, shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
• Deep understanding of multilingual text processing pitfalls.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets and files in your native language.
• Prompting & Translation: identifying failure points in your native language.
• Implementation & Verification: support reference implementations and write reliable verifier scripts.

Skills

Python

Shell scripting

Data processing

CLI workflows

Coding agents

Multilingual processing

Education

Top-tier universities

Job description

LILT is building a rigorous evaluation suite of Terminal-Bench tasks designed to test multilingual robustness across prompt language effects, non-English data processing, and locale/encoding edge cases in terminal workflows.

We seek experienced native-speaking software engineers to design, build, and validate these benchmarks. You will create high-signal tasks that test a model’s native-language handling without relying on English translation. This is a remote, freelance opportunity.