Remote Native Spanish AI Benchmark Engineer

LILT; Madrid; A distancia; EUR 60.000 - 90.000; Jornada completa

Descripción de la vacante

LILT is seeking experienced software engineers to design, build, and validate a multilingual evaluation suite for terminal workflows. You will create high-signal tasks in your native language, ensure assets remain in the target language, and develop deterministic verifier scripts.

This is a remote, freelance opportunity with flexible scheduling. Responsibilities include engineering tasks, asset creation, translation, implementation, calibration, and rigorous quality control across stages to

Formación

• 5+ years of industry experience in software engineering.
• Proven track record at leading technology companies or graduation from top-tier engineering universities.
• Native or near-native fluency with excellent English grammar, register and phrasing.
• Strong proficiency in Python, shell scripting, and data processing.
• Extensive experience with Terminal/CLI workflows and coding agents.

Responsabilidades

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build multilingual task environments in native languages.
• Prompting & Translation: identify failure points in native language.
• Implementation & Verification: develop reference implementations and reliable verifier scripts.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in a 4-layer quality process with automated checks.

Conocimientos

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

Unicode normalization

Descripción del empleo

LILT is seeking experienced software engineers to design, build, and validate a multilingual evaluation suite for terminal workflows. You will create high-signal tasks in your native language, ensure assets remain in the target language, and develop deterministic verifier scripts.

This is a remote, freelance opportunity with flexible scheduling. Responsibilities include engineering tasks, asset creation, translation, implementation, calibration, and rigorous quality control across stages to