Software Engineers: Paid Interview on AI Evaluation Tasks

Triwill Group; Northern (KY); Hybrid; USD 207,000 - 482,000; Full time

Job summary

Triwill Group, through a partner company, is seeking Software Engineers for a paid interview on AI evaluation tasks. This is a fully remote opportunity based in the United States.

You will review coding challenges and evaluation environments, assess task accuracy, challenge level, and realism, analyze evaluation harnesses, and provide objective feedback to improve AI testing standards. Candidates should articulate reasoning clearly.

Qualifications

• Experience reviewing coding challenges and evaluation environments.
• Ability to identify issues in task design and fairness.
• Capability to articulate reasoning and provide structured feedback.

Responsibilities

• Review coding challenges and evaluation environments used to benchmark AI agents.
• Examine evaluation harnesses and walk through their logic to spot issues.
• Provide objective feedback to improve realism and technical accuracy of tasks.

Skills

Code evaluation

Technical reasoning

Clear explanations

Problem solving

Tools

Evaluation harness analysis

Job description

Software Engineers: Paid Interview on AI Evaluation Tasks

Remote US

Skills

Software Engineering Programming Tasks Evaluation Technical Systems Assessment Code Quality Assessment AI Performance Evaluation Evaluation Harness Examination Logic Analysis Problem-Solving Communication Collaboration Technical Feedback Real-World Engineering Standards

Description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for Software Engineers: Paid Interview on AI Evaluation Tasks based in the United States.

This is a remote, paid research opportunity for software engineers with hands-on experience evaluating realistic programming tasks and technical systems.
You will review coding challenges and the evaluation environments used to assess the performance of AI agents.
Your expertise will help determine whether these tasks are technically accurate, appropriately challenging, verifiable, and representative of real-world engineering standards.
You will examine evaluation harnesses, walk through their logic, and identify potential technical or structural issues.
Your feedback will contribute to improving how AI systems are tested and benchmarked against practical software engineering expectations.
The session is designed for experienced technical professionals who can clearly explain their reasoning and assess code quality objectively.
This is a flexible opportunity to apply your engineering expertise to the development of more rigorous and realistic AI evaluations.

More openings like Software Engineers: Paid Interview on AI Evaluation Tasks at Jobgether and related companies.

SEO landing pages for popular locations, roles, and remote work.