Remote German AI Benchmark Engineer (Native Speaker)

lilt-production; Österreich; Remote; EUR 60.000 - 90.000; Vollzeit

Zusammenfassung

Lilt is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks for evaluating language models. This remote, freelance opportunity focuses on creating high-quality tasks in your native language, preserving linguistic nuance without English translation crutches.

You'll deliver task environments, prompts, and robust verifier scripts, with a strict four-layer quality-control process and exposure to cutting-edge AI and language technology across

Qualifikationen

• 5+ years of industry experience in software engineering.
• Native or near-native fluency with high English proficiency.
• Strong Python, shell scripting, and data processing skills.
• Extensive experience with Terminal/CLI-based development workflows.
• Deep understanding of multilingual text processing pitfalls, including encoding/decoding, locale conventions and safe string operations.

Aufgaben

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build native-language task environments with real data.
• Prompting & Translation: identify failure points in your language.
• Implementation & Verification: support reference implementations and deterministic verifier scripts.
• Calibration & Execution: analyze logs and tune task difficulty across model tiers.
• Quality Assurance: participate in 4-layer review process and automated checks.

Kenntnisse

Python

Shell scripting

Data processing

CLI workflows

Coding agents

Multilingual NLP

Unicode handling

Jobbeschreibung

Lilt is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks for evaluating language models. This remote, freelance opportunity focuses on creating high-quality tasks in your native language, preserving linguistic nuance without English translation crutches.

You'll deliver task environments, prompts, and robust verifier scripts, with a strict four-layer quality-control process and exposure to cutting-edge AI and language technology across