Remote AI Coding Evaluator — Software Engineering QA

agency; United States; Remote; USD 11,000 - 90,000; Full time

Benefits offered by this job

Remote contract work

Job summary

Agency is seeking a contractor to help evaluate and improve an AI model’s software engineering knowledge. You will test coding tasks and CS questions, judge accuracy and logic, and document errors with clear examples.

Responsibilities include assessing topics from data structures to secure coding, and recording failure cases to improve prompts and evaluation criteria. Remote contract work with rate based on experience.

Qualifications

• Strong knowledge of programming and software development.
• Clear communication and ability to explain reasoning step by step.
• Degree in computer science or closely related field preferred.

Responsibilities

• Explore coding tasks covering software engineering and CS.
• Assess responses on data structures, algorithms, databases, systems programming, and web development.
• Test topics such as object-oriented design, Git, REST APIs, debugging, data pipelines, and secure coding.
• Verify factual accuracy and logical soundness of proposed solutions.
• Record reproducible failure cases and suggest prompt/evaluation improvements.

Skills

Programming knowledge

Clear communication

Evaluation of technical answers

Education

Degree in Computer Science or related field

Preferred: professional development/open-source/competitive programming

Tools

Git

API design

Data pipelines

Job description

Agency is seeking a contractor to help evaluate and improve an AI model’s software engineering knowledge. You will test coding tasks and CS questions, judge accuracy and logic, and document errors with clear examples.

Responsibilities include assessing topics from data structures to secure coding, and recording failure cases to improve prompts and evaluation criteria. Remote contract work with rate based on experience.