Remote Native-Language AI Benchmark Engineer

Lilt; Argentina; A distancia; ARS 124.818.000 - 249.637.000; A tiempo parcial

Ventajas ofrecidas por este puesto de trabajo

Remote work

Flexible schedule

Prompt payments

Descripción de la vacante

LILT is seeking experienced software engineers to design, build, and validate benchmarks for multilingual models. This remote, freelance role focuses on creating high-quality tasks in the candidate’s native language, evaluating agents, and writing deterministic verifier scripts.

You’ll work across data, prompts, and evaluation rubrics, collaborating with a global linguistics and ML team. Ideal candidates have 5+ years of software engineering experience, strong Python and shell skills, and a deep

Formación

• 5+ years of software engineering experience.
• Proven track record at leading tech companies or top-tier universities.
• Native or near-native fluency in the target language with high English proficiency.
• Strong proficiency in Python, shell scripting, and data processing.

Responsabilidades

• Task Engineering: evaluating coding agents.
• Asset Creation: build datasets and files in the native language.
• Prompting & Translation: identify failure points in the native language.
• Implementation & Verification: develop reference implementations and verifier scripts.
• Calibration & Execution: analyze logs and calibrate task difficulty across model tiers.
• Quality Assurance: participate in a 4-layer QA process.

Conocimientos

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual text processing

Unicode handling

Educación

Engineering degree

Descripción del empleo

LILT is seeking experienced software engineers to design, build, and validate benchmarks for multilingual models. This remote, freelance role focuses on creating high-quality tasks in the candidate’s native language, evaluating agents, and writing deterministic verifier scripts.

You’ll work across data, prompts, and evaluation rubrics, collaborating with a global linguistics and ML team. Ideal candidates have 5+ years of software engineering experience, strong Python and shell skills, and a deep