Remote Python Code QA for AI Evaluation

CNTXT AI; Abu Dhabi; Remote; AED 202,000 - 405,000; Part time

Benefits offered by this job

Remote work

Flexible schedule

Job summary

CNTXT AI is seeking a remote hourly contractor to evaluate AI-generated Python code and develop cases that test coding accuracy and reasoning quality. You will assess correctness, efficiency, readability, and adherence to real-world engineering standards, identifying bugs and edge cases while documenting clear corrections.

Ideal candidates have a CS/SE background, strong Python skills, and the ability to write precise evaluation notes in professional English.

Qualifications

• Background in Computer Science, Software Engineering, or related field preferred.
• Proficiency in Python, including data structures, algorithms, libraries, performance optimization, and idiomatic coding practices.
• Ability to read, write, and critically evaluate Python code across varying complexity levels.
• Full professional English proficiency.

Responsibilities

• Evaluate AI-generated Python code for correctness, efficiency, readability, and adherence to real-world engineering standards.
• Identify bugs, logical errors, edge cases, and anti-patterns in AI outputs, and explain corrections clearly in writing.
• Apply structured evaluation rubrics to score code quality consistently across tasks.
• Develop prompts and test cases that probe AI accuracy across Python programming tasks.
• Rate and compare AI-generated solutions based on correctness, internal consistency, and adherence to the prompt.

Skills

Python

Code review

Test design

Technical writing

Education

Bachelor's degree in CS/SE

Tools

Git

pytest

Job description

CNTXT AI is seeking a remote hourly contractor to evaluate AI-generated Python code and develop cases that test coding accuracy and reasoning quality. You will assess correctness, efficiency, readability, and adherence to real-world engineering standards, identifying bugs and edge cases while documenting clear corrections.

Ideal candidates have a CS/SE background, strong Python skills, and the ability to write precise evaluation notes in professional English.