Remote AI Benchmark Engineer (Multilingual)

LILT; United States; Remote; USD 96,000 - 165,000; Full time

Benefits offered by this job

Remote work

Flexible schedule

Competitive rates

Prompt payments

Job summary

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks. You will create high-signal tasks in your native language and keep assets in the target language to measure multilingual handling without English crutches.

This remote contract role offers flexible scheduling, competitive rates, prompt payments, and a chance to shape cutting-edge AI and language technology while collaborating with a global community of linguists and engineers

Qualifications

• 5+ years of industry experience in software engineering.
• Background with leading technology companies or top-tier universities.
• Native or near-native fluency, with high English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
• Deep technical understanding of multilingual text processing pitfalls, including encoding/decoding robustness, Unicode normalization, and locale conventions.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets and files in your native language.
• Prompting & Translation: finding failure points where AI does not work, in your native language.
• Implementation & Verification: Support the development of robust solutions and write reliable verifier scripts.
• Calibration & Execution: Analyze execution logs and calibrate task difficulty using Terminal-Bench configurations.
• Quality Assurance: Participate in a 4-layer human quality control process and automated checks.

Skills

5+ years experience

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

Coding agents

English proficiency

Job description

LILT is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks. You will create high-signal tasks in your native language and keep assets in the target language to measure multilingual handling without English crutches.

This remote contract role offers flexible scheduling, competitive rates, prompt payments, and a chance to shape cutting-edge AI and language technology while collaborating with a global community of linguists and engineers