Remote Freelance AI Benchmark Engineer — Arabic

LILT; United States; Remote; USD 83,000 - 152,000; Part time

Benefits offered by this job

Remote freelance opportunity

Competitive rates

Job summary

LILT is seeking experienced software engineers to design and validate multilingual benchmarks, building tasks that test models in native languages without English crutches. This is a remote, freelance opportunity with flexible scheduling and competitive rates.

You will create high-signal tasks using native-language datasets, and develop robust verifier scripts while calibrating task difficulty across model tiers. Join a global team focused on advancing multilingual AI capabilities.

Qualifications

• 5+ years of software engineering experience at leading tech firms or top-tier universities.
• Native or near-native fluency in English with strong proficiency.
• Strong Python, shell scripting, and data processing skills.
• Extensive experience with Terminal/CLI workflows and coding agents.
• Deep understanding of multilingual text processing pitfalls and Unicode handling.

Responsibilities

• Design and validate multilingual benchmarks for encoding and locale handling.
• Create high-signal tasks using native-language datasets.
• Develop reference implementations and robust verifier scripts.
• Analyze logs and calibrate task difficulty across model tiers.
• Participate in a rigorous 4-layer QA process and automated checks.

Skills

Software engineering

Data processing

English proficiency

CLI development

Education

CS/Engineering degree

Tools

Python

Shell scripting

Job description

LILT is seeking experienced software engineers to design and validate multilingual benchmarks, building tasks that test models in native languages without English crutches. This is a remote, freelance opportunity with flexible scheduling and competitive rates.

You will create high-signal tasks using native-language datasets, and develop robust verifier scripts while calibrating task difficulty across model tiers. Join a global team focused on advancing multilingual AI capabilities.