Contract AI Software Evaluator – Code & Benchmarking

Turing

Pittsburgh (Allegheny County)

Remote

USD 55,000 - 110,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Turing, based in San Francisco, seeks a Software Engineering Evaluator to help build and curate data for training frontier AI models. You'll work with Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go, crafting precise solutions and correcting code.

You will evaluate AI-generated code for efficiency, collaborate with researchers and cross-functional teams, and help verify solution quality through automated checks and error pattern identification.

Qualifications

  • 3+ years of software engineering experience.
  • Strong expertise in building full‑stack applications and deploying scalable, production‑grade software.
  • Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
  • Excellent oral and written communication skills for clear evaluation rationales.

Responsibilities

  • Curate code examples and build solutions across Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
  • Evaluate and refine AI-generated code for efficiency, scalability, and reliability.
  • Collaborate with cross-functional teams to improve AI-driven coding solutions.
  • Build agents that verify code quality and identify error patterns.
  • Hypothesize software engineering steps and evaluate model capabilities.
  • Design verification mechanisms to automatically verify solutions.

Skills

Software engineering
Full-stack development
System design
Code quality
Technical communication

Job description

Turing, based in San Francisco, seeks a Software Engineering Evaluator to help build and curate data for training frontier AI models. You'll work with Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go, crafting precise solutions and correcting code.

You will evaluate AI-generated code for efficiency, collaborate with researchers and cross-functional teams, and help verify solution quality through automated checks and error pattern identification.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Software Evaluator, Systems & Benchmarking
AI Software Evaluator, Systems & Benchmarking

AI Trainer Gigs • United States

Remote
USD 83,000 - 165,000
Contract AI Software Engineer: Code Curation & Evaluation
Contract AI Software Engineer: Code Curation & Evaluation

Turing • Chicago (IL)

Remote
USD 55,000 - 110,000
Flexible hours
Partial PST overlap
AI Software Engineer - Data & Code Evaluator (Contractor)
AI Software Engineer - Data & Code Evaluator (Contractor)

Turing • United States

Remote
USD 55,000 - 96,000
Flexible engagement (10–40 hrs/week)
Contractor (no medical/paid leave)
Remote AI Code Evaluator: Full-Stack Engineer
Remote AI Code Evaluator: Full-Stack Engineer

Turing • United States

Remote
USD 55,000 - 96,000
Remote AI Software Evaluator (Python/ML) — Contractor
Remote AI Software Evaluator (Python/ML) — Contractor

Turing • Los Angeles (CA)

Remote
USD 55,000 - 96,000
Senior Software Engineer - AI Code Evaluator
Senior Software Engineer - AI Code Evaluator

Turing • United States

Remote
USD 55,000 - 110,000
Senior AI Software Evaluator — Python & ML Infra
Senior AI Software Evaluator — Python & ML Infra

AI Trainer Gigs • San Francisco (CA)

On-site
USD 83,000 - 138,000
Remote AI Software Evaluator – Full-Stack Engineer (Python)
Remote AI Software Evaluator – Full-Stack Engineer (Python)

Turing • Chicago (IL)

Remote
USD 83,000 - 165,000
Full-Stack AI Code Engineer (Contract, 10–40 hrs/wk)
Full-Stack AI Code Engineer (Contract, 10–40 hrs/wk)

Turing • United States

Remote
USD 83,000 - 138,000
AI Code Evaluation Engineer
AI Code Evaluation Engineer

Turing • Chicago (IL)

Remote
USD 69,000 - 103,000