Arabic AI Benchmark Engineer (Remote, Freelance)

LILT; United States; Remote; USD 83,000 - 152,000; Full time

Benefits offered by this job

Flexible schedule

Competitive rates

Fast payments

Global community

Job summary

LILT seeks experienced software engineers to design, build, and validate multilingual benchmarks in a remote freelance capacity. You will create high-signal tasks in native languages and test model handling without English crutches.

You’ll work on task engineering, asset creation, prompting, and verification, with a rigorous 4-layer QA process and deterministic rubrics to ensure fairness and accuracy.

Qualifications

• 5+ years of software engineering experience.
• Proven track record at leading tech firms or top-tier engineering degree.
• Native or near-native fluency with strong English proficiency.
• Strong Python, shell scripting, and data processing skills.
• Extensive experience with Terminal/CLI workflows and coding agents.
• Deep understanding of multilingual text processing pitfalls (Unicode, locale rules, RTL).

Responsibilities

• Task Engineering: Evaluating Coding Agents.
• Asset Creation: Build native-language task environments with authentic datasets.
• Prompting & Translation: identify failure points in your language.
• Implementation & Verification: develop reference implementations and deterministic verifier scripts.
• Calibration & Execution: analyze logs and adjust task difficulty across model tiers.
• Quality Assurance: participate in 4-layer review and automated checks.

Skills

Python

Shell scripting

Data processing

Multilingual text processing

Terminal/CLI workflows

Coding agents

Education

Bachelor's degree or higher

Tools

CLI tooling

Job description

LILT seeks experienced software engineers to design, build, and validate multilingual benchmarks in a remote freelance capacity. You will create high-signal tasks in native languages and test model handling without English crutches.

You’ll work on task engineering, asset creation, prompting, and verification, with a rigorous 4-layer QA process and deterministic rubrics to ensure fairness and accuracy.