Remote AI Code Evaluator (Contractor)

First Roles; Oregon; Remote; USD 207,000 - 276,000; Part time

Benefits offered by this job

Remote work

Job summary

First Roles is seeking a Junior Software Engineer for a remote contractor role focused on evaluating AI-generated code and creating challenging programming problems. You will develop reference solutions and test cases, review AI models’ reasoning on architecture and debugging, and provide detailed feedback to improve code quality and model performance.

The role requires strong fundamentals, 1+ year of software engineering experience, and the ability to work independently with clear written

Qualifications

• At least one year of professional software engineer experience.
• Strong fundamentals in programming, data structures, and algorithms.
• Experience writing, debugging, testing, and reviewing code.
• Familiarity with version control, code reviews, testing, and debugging.

Responsibilities

• AI-Generated Code Evaluation: review AI-generated code for correctness and quality.
• Coding Problem Development: design challenging programming problems and test cases.
• Reference Solution Creation: develop accurate reference solutions.
• Test Case Development: create comprehensive test cases to validate functionality.
• Technical Reasoning Assessment: evaluate reasoning on architecture, design, debugging.
• Code Quality Analysis: identify bugs, inefficiencies, and security concerns.
• Technical Feedback: provide actionable feedback on errors and improvements.
• Documentation: document requirements, behavior, and evaluation findings.
• Quality Assurance: ensure tasks and criteria are reproducible.

Skills

Software fundamentals

Programming

Data structures

Algorithms

Code review

Debugging

Testing

Documentation

Problem solving

Tools

Git

CI/CD

Unit testing

Code reviews

Job description

First Roles is seeking a Junior Software Engineer for a remote contractor role focused on evaluating AI-generated code and creating challenging programming problems. You will develop reference solutions and test cases, review AI models’ reasoning on architecture and debugging, and provide detailed feedback to improve code quality and model performance.

The role requires strong fundamentals, 1+ year of software engineering experience, and the ability to work independently with clear written