Remote Arabic AI Benchmark Engineer (Native)

LILT; Abu Dhabi; Remote; AED 202,000 - 405,000; Full time

Benefits offered by this job

Flexible schedule

Prompt payments

Work on impactful AI benchmarks

Global community of language experts

Job summary

LILT is building a rigorous verifiable evaluation suite to test multilingual software challenges and model robustness across non-English data processing. We seek experienced native-speaking software engineers to design, build, and validate benchmarks that avoid English translation crutches.

This remote freelance opportunity focuses on creating high-signal tasks, translating prompts, and developing reliable verifier scripts while participating in a multi-layer QA process.

Qualifications

• 5+ years of industry software engineering experience.
• Native or near-native fluency with high English proficiency.
• Strong proficiency in Python, shell scripting and data processing.

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build realistic task environments using datasets in your native language.
• Prompting & Translation: identify failure points where AI struggles in your native language.
• Implementation & Verification: develop robust reference implementations and deterministic verifier scripts.
• Calibration & Execution: analyze logs and tune task difficulty across model tiers.
• Quality Assurance: participate in 4-layer human QA plus automated checks for fairness and accuracy.

Skills

Python

Shell scripting

Data processing

Terminal/CLI workflows

Multilingual NLP

Education

Engineering degree from a top-tier university

Job description

LILT is building a rigorous verifiable evaluation suite to test multilingual software challenges and model robustness across non-English data processing. We seek experienced native-speaking software engineers to design, build, and validate benchmarks that avoid English translation crutches.

This remote freelance opportunity focuses on creating high-signal tasks, translating prompts, and developing reliable verifier scripts while participating in a multi-layer QA process.