Remote AI Benchmark Engineer — Indonesian Language Expert

Lilt; Tangerang; Hybrid; IDR 1,457,264,000 - 2,428,773,000; Part time

Benefits offered by this job

Remote freelance work

Competitive rates

Flexible schedule

Job summary

Lilt is seeking experienced software engineers to design and validate multilingual benchmarks. This remote freelance role focuses on building high-quality tasks and environments in native languages to measure multilingual robustness without English translation aids.

You will work with a global team to test prompts, translate where appropriate, and develop reliable verifier scripts. The role emphasizes quality control and exposure to diverse model tiers across languages.

Qualifications

• 5+ years of industry experience in software engineering.
• Native or near-native fluency with deep understanding of grammar, register, and phrasing rules; high English proficiency.
• Background at leading tech companies or top-tier universities is preferred.
• Strong proficiency in Python, shell scripting, and data processing.
• Experience with Terminal/CLI workflows and coding agents.
• Deep understanding of multilingual text processing pitfalls (Unicode, locale, RTL handling).

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build task environments in native languages, keeping assets in the target language.
• Prompting & Translation: identify failure points in your language.
• Implementation & Verification: support reference implementations and deterministic verifier scripts.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in 4-layer QA process and automated checks.

Skills

Multilingual proficiency

Native or near-native fluency

Strong English proficiency

Education

Bachelor's degree in CS/Software Eng

Tools

Python

Shell scripting

Data processing

Job description

Lilt is seeking experienced software engineers to design and validate multilingual benchmarks. This remote freelance role focuses on building high-quality tasks and environments in native languages to measure multilingual robustness without English translation aids.

You will work with a global team to test prompts, translate where appropriate, and develop reliable verifier scripts. The role emphasizes quality control and exposure to diverse model tiers across languages.