Remote AI Benchmark Engineer – Spanish (Mexico) Expert

lilt-production; Mexico; Remote; PHP 4,195,000 - 6,992,000; Part time

Job summary

Lilt is building a rigorous, verifiable evaluation suite of Terminal-Bench tasks to test multilingual software challenges. We seek experienced native-speaking software engineers to design, build, and validate benchmarks that measure multilingual robustness without relying on English translation.

As a remote freelance contractor, you will craft authentic tasks in your native language, develop reference implementations, write deterministic verifier scripts, calibrate task difficulty, and

Qualifications

• 5+ years of industry software engineering experience.
• Native or near-native fluency in the target language with strong English proficiency.
• Strong Python, shell scripting, and data processing skills.
• Extensive experience with Terminal/CLI-based development workflows and coding agents.
• Deep understanding of multilingual text processing pitfalls (encoding, locale rules, RTL handling).

Responsibilities

• Design, build and validate multilingual benchmarks.
• Create high-signal tasks in your native language.
• Identify failure points in prompts in your language.
• Develop reference implementations and deterministic verifier scripts.
• Calibrate task difficulty and run configurations across model tiers.
• Participate in a 4-layer quality control process and automated checks.

Skills

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

Unicode handling

Education

Top-tier engineering university graduation

Tools

CLI tools

Job description

Lilt is building a rigorous, verifiable evaluation suite of Terminal-Bench tasks to test multilingual software challenges. We seek experienced native-speaking software engineers to design, build, and validate benchmarks that measure multilingual robustness without relying on English translation.

As a remote freelance contractor, you will craft authentic tasks in your native language, develop reference implementations, write deterministic verifier scripts, calibrate task difficulty, and