Remote AI Benchmark Engineer & Native Chinese Specialist

Hidden Jobs; United States; Remote; USD 83,000 - 165,000; Full time

Benefits offered by this job

Fully remote

1099 contractor

Flexible scheduling

Prompt payment

Job summary

Hidden Jobs is seeking native Chinese-speaking software engineers for a remote freelance role to design multilingual evaluation benchmarks for large language models. You will craft Terminal-Bench tasks, use Chinese-language datasets, and implement deterministic verifiers to ensure robust evaluation across encoding and locale scenarios.

Candidates should have 5+ years in software engineering, experience at top tech firms or universities, and strong Chinese/English fluency with Python and shell

Qualifications

• 5+ years of professional software engineering experience.
• Experience at leading technology companies or top-tier universities preferred.
• Native or near-native Chinese fluency with strong English proficiency.

Responsibilities

• Design and engineer Terminal-Bench tasks that evaluate coding agents in Chinese-language scenarios.
• Build realistic task environments using Chinese-language datasets and files, keeping assets in the target language.
• Identify failure points where AI behavior degrades in Chinese-only contexts.
• Develop reference implementations and write deterministic verifier scripts, using rubric-based judging only when strictly necessary.
• Calibrate task difficulty by analyzing execution logs and running benchmarks against multiple model tiers.
• Participate in a multi-layer human quality control process alongside automated LLM-based checks.

Skills

Python

Shell scripting

Terminal/CLI development

Unicode handling

Multilingual data processing

Education

Bachelor's degree in Computer Science or related field

Tools

Git

Job description

Hidden Jobs is seeking native Chinese-speaking software engineers for a remote freelance role to design multilingual evaluation benchmarks for large language models. You will craft Terminal-Bench tasks, use Chinese-language datasets, and implement deterministic verifiers to ensure robust evaluation across encoding and locale scenarios.

Candidates should have 5+ years in software engineering, experience at top tech firms or universities, and strong Chinese/English fluency with Python and shell