Remote Senior Software Engineer - AI Training & Evaluation

Benture; United States; Remote; USD 165,000 - 248,000; Full time

Job summary

Mercor is collaborating with a leading AI research lab to design and evaluate frontier AI-model tasks in software engineering. The role requires deep engineering experience from top tech, quant, or financial firms and strong abilities in languages such as Python, Java, C++, Go, TypeScript, or Rust.

You will design tasks, write reference solutions and tests, and assess AI-generated work with senior reviewers to ensure rigorous, reproducible evaluations.

Qualifications

• 4+ years of professional software engineering experience in leading tech/quant firm or top financial institution
• Strong command of at least one major language (Python/Java/C++/Go/TypeScript/Rust) with system design
• Proven ability to work in large, complex production codebases and to judge code quality beyond tests
• Excellent written English and ability to explain technical decisions clearly
• Bachelor's degree from an English-speaking country
• Detail-oriented, consistent approach to evaluating complex work
• Comfortable receiving feedback and calibrating judgment against standards

Responsibilities

• Design realistic software engineering tasks drawn from real-world work across large codebases
• Write clear task proposals explaining realism, verification, and difficulty for AI models
• Write reference solutions, tests and precise criteria for correctness and maintainability
• Assess AI-generated code and engineering work with senior reviewers to judge success and failure
• Apply evidence-based judgment to ensure assessments are reproducible and defensible
• Take structured feedback from senior reviewers and iterate on submitted work

Skills

Python

Java

C++

Go

TypeScript

Rust

System design

Testing

Debugging

Code review

English proficiency

Education

Bachelor's degree from an English-speaking country

Job description

Mercor is collaborating with a leading AI research lab to design and evaluate frontier AI-model tasks in software engineering. The role requires deep engineering experience from top tech, quant, or financial firms and strong abilities in languages such as Python, Java, C++, Go, TypeScript, or Rust.

You will design tasks, write reference solutions and tests, and assess AI-generated work with senior reviewers to ensure rigorous, reproducible evaluations.