Remote Native-Language AI Benchmark Engineer

LILT; United States; Remote; USD 83,000 - 179,000; Full time

Benefits offered by this job

Flexible schedule

Fast payments

Global language projects

Community of linguists

Job summary

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks. You will create high-signal tasks in your native language to test model handling of multilingual environments without relying on English translation.

This is a remote freelance opportunity with flexible scheduling. Responsibilities include task engineering, asset creation in the target language, prompting, verification, calibration, and rigorous quality assurance across multiple

Qualifications

• 5+ years of industry experience in software engineering.
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
• Native or near-native fluency, with a deep understanding of its grammar, register, and phrasing rules. High English proficiency.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets and files in your native language, maintaining the target language.
• Prompting & Translation: identify failure points in your native language.
• Implementation & Verification: contribute reference implementations and deterministic verifier scripts when needed.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in a 4-layer human QC process and automated checks.

Skills

Software engineering experience

Python

Shell scripting

Data processing

Terminal/CLI workflows

Education

Bachelor's degree or equivalent

Job description

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks. You will create high-signal tasks in your native language to test model handling of multilingual environments without relying on English translation.

This is a remote freelance opportunity with flexible scheduling. Responsibilities include task engineering, asset creation in the target language, prompting, verification, calibration, and rigorous quality assurance across multiple