AI Code Evaluator for Frontier Models (Contractor)

Turing; Austin (TX); On-site; USD 83,000 - 138,000; Full time

Job summary

Turing seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models, focusing on Python, C/C++, Rust, Go, Java, and JavaScript (ReactJS). You will work with researchers to improve enterprise AI-driven coding solutions.

The role emphasizes evaluating AI-generated code for efficiency and reliability, building verification mechanisms, and collaborating with cross-functional teams. This is a contractor role based in the United States.

Qualifications

• Several years of software engineering experience (3+ years).
• Strong expertise in systems programming, infrastructure, or backend development using languages like Python, C/C++, Rust, and Go.
• Experience building and deploying scalable, production-grade software.
• Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
• Excellent oral and written communication skills for clear, structured evaluation rationales.

Responsibilities

• Curate code examples, provide precise solutions, and correct code in Python, C/C++, Rust, Go, Java, and JavaScript (ReactJS).
• Evaluate and refine AI-generated code for efficiency, scalability, and reliability.
• Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
• Build agents that verify the quality of systems-level and infrastructure code and identify error patterns.
• Hypothesize on steps in the software engineering cycle and evaluate model capabilities on them.

Skills

Python

C/C++

Rust

Go

Backend development

Systems programming

Communication skills

Tools

ReactJS

Job description

Turing seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models, focusing on Python, C/C++, Rust, Go, Java, and JavaScript (ReactJS). You will work with researchers to improve enterprise AI-driven coding solutions.

The role emphasizes evaluating AI-generated code for efficiency and reliability, building verification mechanisms, and collaborating with cross-functional teams. This is a contractor role based in the United States.