AI Software Evaluator, Systems & Benchmarking

AI Trainer Gigs

United States

Remote

USD 83,000 - 165,000

Part time

13 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Turing, based in San Francisco, seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models, collaborating closely with researchers. This role focuses on curating code examples, providing precise solutions, and making corrections in Python, C/C++, Rust, Go, Java, and JavaScript (ReactJS), with emphasis on systems-level code and infrastructure.

You will evaluate and refine AI-generated code for efficiency, scalability, and

Qualifications

  • 3+ years of software engineering experience
  • Strong expertise in systems programming, infrastructure, or backend development
  • Experience building and deploying scalable, production-grade software
  • Deep understanding of software architecture, design, development, debugging, and code quality/review assessment
  • Excellent oral and written communication skills for clear, structured evaluation rationales

Responsibilities

  • Curate code examples for AI model training and benchmarks
  • Evaluate and refine AI-generated code for performance and reliability
  • Collaborate with researchers and cross-functional teams on coding solutions
  • Build agents to verify system-level and infrastructure code quality
  • Prototyping, architecture design, API design, production implementation, monitoring

Skills

Python
C/C++
Rust
Go
Java
JavaScript/ReactJS
Software engineering experience
Code quality & review
Systems programming
Backend development
Communication skills

Job description

Turing, based in San Francisco, seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models, collaborating closely with researchers. This role focuses on curating code examples, providing precise solutions, and making corrections in Python, C/C++, Rust, Go, Java, and JavaScript (ReactJS), with emphasis on systems-level code and infrastructure.

You will evaluate and refine AI-generated code for efficiency, scalability, and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Contract AI Software Evaluator – Code & Benchmarking
Contract AI Software Evaluator – Code & Benchmarking

Turing • Pittsburgh

Remote
USD 55,000 - 110,000
Senior AI Software Evaluator — Python & ML Infra
Senior AI Software Evaluator — Python & ML Infra

AI Trainer Gigs • San Francisco (CA)

On-site
USD 83,000 - 138,000
AI Software Engineer - Data & Code Evaluator (Contractor)
AI Software Engineer - Data & Code Evaluator (Contractor)

Turing • United States

Remote
USD 55,000 - 96,000
Flexible engagement (10–40 hrs/week)
Contractor (no medical/paid leave)
Remote AI Software Evaluator (Python/ML) — Contractor
Remote AI Software Evaluator (Python/ML) — Contractor

Turing • Los Angeles (CA)

Remote
USD 55,000 - 96,000
Contract AI Software Engineer: Code Curation & Evaluation
Contract AI Software Engineer: Code Curation & Evaluation

Turing • Chicago (IL)

Remote
USD 55,000 - 110,000
Flexible hours
Partial PST overlap
Remote AI Code Evaluator: Full-Stack Engineer
Remote AI Code Evaluator: Full-Stack Engineer

Turing • United States

Remote
USD 55,000 - 96,000
Senior Python AI Evaluation Engineer
Senior Python AI Evaluation Engineer

AI Trainer Gigs • San Francisco (CA)

Hybrid
USD 55,000 - 110,000
Remote AI Software Evaluator – Full-Stack Engineer (Python)
Remote AI Software Evaluator – Full-Stack Engineer (Python)

Turing • Chicago (IL)

Remote
USD 83,000 - 165,000
AI-Driven Full-Stack Engineer — Production Systems
AI-Driven Full-Stack Engineer — Production Systems

Turing • Pittsburgh

On-site
USD 69,000 - 117,000
AI Code Evaluation Engineer – Frontend & Backend
AI Code Evaluation Engineer – Frontend & Backend

HumanitApp • San Francisco (CA)

On-site
USD 83,000 - 138,000