Remote AI Benchmark Engineer & Native Chinese Specialist

lilt-production

United States

Remote

USD 83,000 - 165,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote
1099 contractor
Flexible scheduling
Prompt payment

Job summary

Hidden Jobs is seeking native Chinese-speaking software engineers for a remote freelance role to design multilingual evaluation benchmarks for large language models. You will craft Terminal-Bench tasks, use Chinese-language datasets, and implement deterministic verifiers to ensure robust evaluation across encoding and locale scenarios.

Candidates should have 5+ years in software engineering, experience at top tech firms or universities, and strong Chinese/English fluency with Python and shell

Qualifications

  • 5+ years of professional software engineering experience.
  • Experience at leading technology companies or top-tier universities preferred.
  • Native or near-native Chinese fluency with strong English proficiency.

Responsibilities

  • Design and engineer Terminal-Bench tasks that evaluate coding agents in Chinese-language scenarios.
  • Build realistic task environments using Chinese-language datasets and files, keeping assets in the target language.
  • Identify failure points where AI behavior degrades in Chinese-only contexts.
  • Develop reference implementations and write deterministic verifier scripts, using rubric-based judging only when strictly necessary.
  • Calibrate task difficulty by analyzing execution logs and running benchmarks against multiple model tiers.
  • Participate in a multi-layer human quality control process alongside automated LLM-based checks.

Skills

Python
Shell scripting
Terminal/CLI development
Unicode handling
Multilingual data processing

Education

Bachelor's degree in Computer Science or related field

Tools

Git

Job description

Hidden Jobs is seeking native Chinese-speaking software engineers for a remote freelance role to design multilingual evaluation benchmarks for large language models. You will craft Terminal-Bench tasks, use Chinese-language datasets, and implement deterministic verifiers to ensure robust evaluation across encoding and locale scenarios.

Candidates should have 5+ years in software engineering, experience at top tech firms or universities, and strong Chinese/English fluency with Python and shell

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Benchmark Engineer — Multilingual Native Specialist
Remote AI Benchmark Engineer — Multilingual Native Specialist

Lilt, Inc. • United States

Remote
USD 70,000 - 120,000
Remote Multilingual AI Benchmark Engineer (Freelance)
Remote Multilingual AI Benchmark Engineer (Freelance)

Lilt, Inc. • United States

Remote
EUR 70,000 - 100,000
Flexible schedule
Prompt payments
Impactful projects
+3
Remote Technical SME & AI Benchmarking Engineer
Remote Technical SME & AI Benchmarking Engineer

Lilt, Inc. • United States

Remote
USD 83,000 - 165,000
Remote AI Benchmark Engineer: German (Austria) Expert
Remote AI Benchmark Engineer: German (Austria) Expert

lilt-production • United States

Remote
USD 78,000 - 134,000
Flexible hours
Prompt payment
Fully remote freelance engagement
+1
Remote AI Training Engineer (Chinese Fluent)
Remote AI Training Engineer (Chinese Fluent)

Meridial • United States

Remote
USD 11,000 - 90,000
Bilingual AI SME & Engineer — Domain Specialist
Bilingual AI SME & Engineer — Domain Specialist

Lilt, Inc. • United States

Remote
USD 83,000 - 138,000
Flexible schedule
Competitive payments
Meaningful AI projects
+3
Remote AI Coding Specialist for Training (Traditional Chinese)
Remote AI Coding Specialist for Training (Traditional Chinese)

Outlier AI, Inc. • United States

Remote
MXN 582,000 - 1,163,000
Remote work
Flexible hours
Remote AI Trainer: Chinese-Speaking Coding Specialist
Remote AI Trainer: Chinese-Speaking Coding Specialist

Invisible Technologies Inc. • United States

Remote
USD 11,000 - 90,000
Remote Software Engineer: AI Benchmarking & New Benchmarks
Remote Software Engineer: AI Benchmarking & New Benchmarks

Remote • San Francisco (CA)

Remote
USD 125,000 - 200,000
Remote work
Generous PTO
Staff retreats
+1
AI Benchmark Engineer | Native Language Specialist - German (Austria) - Remote
AI Benchmark Engineer | Native Language Specialist - German (Austria) - Remote

lilt-production • United States

Remote
USD 78,000 - 134,000
Flexible hours
Prompt payment
Fully remote freelance engagement
+1