Remote Multilingual AI Benchmark Engineer

LILT (Production); Indonesia; Remote; IDR 620,429,000 - 1,489,029,000; Full time

Job summary

LILT (Production) seeks experienced software engineers to design and validate multilingual benchmarks. You will create high-signal tasks in your native language and test model robustness across non-English data and locale encodings. This is a remote, freelance opportunity designed for experts in multilingual software challenges.

The role emphasizes asset creation, evaluation, and deterministic verifier scripts, with a rigorous 4-layer quality control process to ensure benchmark integrity.

Qualifications

• Experience: 1+ years in software engineering.
• Background: Proven track record at leading technology companies or graduation from top-tier engineering universities.
• Language: Native or near-native fluency, with a deep understanding of its grammar.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets and files in your native language, assets remain in the target language.
• Prompting & Translation: identify failure points where AI does not work, in your native language.
• Implementation & Verification: support robust solutions and write deterministic verifier scripts when needed.
• Calibration & Execution: analyze execution logs and calibrate task difficulty across model tiers.
• Quality Assurance: participate in four-layer quality control and automated checks.

Skills

Software engineering

Education

Top-tier engineering university

Job description

LILT (Production) seeks experienced software engineers to design and validate multilingual benchmarks. You will create high-signal tasks in your native language and test model robustness across non-English data and locale encodings. This is a remote, freelance opportunity designed for experts in multilingual software challenges.

The role emphasizes asset creation, evaluation, and deterministic verifier scripts, with a rigorous 4-layer quality control process to ensure benchmark integrity.