Remote AI Code Benchmark Engineer

Turing

United States

Remote

USD 83,000 - 152,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote
Cutting-edge AI projects

Job summary

Turing is seeking experienced Software Engineers to evaluate and improve AI coding systems. You will review AI-generated code, validate solutions, and help build evaluation datasets and benchmarks. This remote freelance role emphasizes code review, debugging, and applying software engineering judgment to real-world tasks.

The position focuses on measuring and improving coding performance in frontier AI models, with collaboration across project teams and the creation of robust grading rubrics.

Qualifications

  • Bachelor's or Master's degree in CS, Software Engineering, or a related field.
  • 3+ years of professional software engineering experience.
  • Strong proficiency in Python, Java, C/C++, Go, Swift, Objective-C, PHP or SQL.
  • Strong understanding of data structures, algorithms, and debugging.
  • Experience performing code reviews and evaluating code quality in large codebases.
  • Familiarity with Git and modern development workflows.
  • Experience with AI/ML data annotation, NLP, or model evaluation is a plus.

Responsibilities

  • Review and evaluate AI-generated code for correctness, efficiency, and maintainability.
  • Analyze software tasks and validate solutions meet expected outcomes.
  • Debug code, reproduce issues, and verify fixes across environments.
  • Assess model explanations and implementations for technical accuracy.
  • Create, refine, and maintain evaluation datasets, benchmarks, and rubrics.
  • Identify edge cases and failure modes where AI struggles with software tasks.
  • Document findings clearly and provide structured feedback to improve quality.
  • Collaborate with teams to set quality standards and evaluation methodologies.

Skills

Python
Java
C/C++
Go
Swift
SQL
Data structures
Algorithms
Code review
Debugging
Git

Education

Bachelor's or Master's in CS/Software Eng

Tools

Docker
Git

Job description

Turing is seeking experienced Software Engineers to evaluate and improve AI coding systems. You will review AI-generated code, validate solutions, and help build evaluation datasets and benchmarks. This remote freelance role emphasizes code review, debugging, and applying software engineering judgment to real-world tasks.

The position focuses on measuring and improving coding performance in frontier AI models, with collaboration across project teams and the creation of robust grading rubrics.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Coding Evaluator – Software Quality
Remote AI Coding Evaluator – Software Quality

Turing • United States

Remote
USD 3,300 - 5,000
Fully remote
Cutting-edge AI projects
Remote AI Code Evaluator & DevOps Engineer
Remote AI Code Evaluator & DevOps Engineer

Turing • United States

Remote
USD 11,000 - 22,000
Fully remote
Freelance contractor role
Remote AI Code Evaluation Engineer
Remote AI Code Evaluation Engineer

RemoExperts • United States

Remote
USD 90,000 - 145,000
AI Code Evaluation Engineer (Contract)
AI Code Evaluation Engineer (Contract)

Turing • Pittsburgh

Remote
USD 55,000 - 96,000
Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)

Turing • United States

Remote
USD 83,000 - 152,000
Fully remote
Cutting-edge AI projects
Senior Software Engineer, AI Evaluation & Coding Agents
Senior Software Engineer, AI Evaluation & Coding Agents

24 Mag • New York (NY)

Remote
USD 138,000 - 207,000
Remote AI Research & Code Evaluation Engineer
Remote AI Research & Code Evaluation Engineer

OpenTrain AI, Inc. • United States

Remote
USD 55,000 - 165,000
Remote C/C++ AI Training & Code Evaluation Engineer
Remote C/C++ AI Training & Code Evaluation Engineer

HumanitApp • United States

Remote
USD 83,000 - 138,000
Remote AI Code Evaluator - Junior Software Engineer
Remote AI Code Evaluator - Junior Software Engineer

Our Client • United States

Remote
USD 207,000 - 276,000
Remote contract
Competitive hourly rate
Collaborate with AI researchers
Remote C++ Bench Engineer for AI Code Evaluation
Remote C++ Bench Engineer for AI Code Evaluation

Turing • United States

Remote
USD 46,000 - 68,000
Fully remote