Remote AI Coding Evaluator & Task Designer

Sovrano AI; United States; Remote; USD 55,000 - 172,000; Full time

Job summary

Sovrano AI is seeking independent contractors for fully remote, project-based work evaluating AI coding agents. You will create tasks from your own software engineering experience, assign them to frontier AI models, and judge the results as a professional reviewer.

You set your own schedule and will be paid hourly, with $40–$125 per hour and a guaranteed minimum of 10 hours weekly. Pay is processed via PayPal and you are responsible for your own taxes as an independent contractor.

Qualifications

• Bachelor's degree or higher, completed or in progress.
• 2+ years of professional software engineering experience preferred.
• Based in the United States, United Kingdom, Ireland, Canada, or Australia.

Responsibilities

• Build realistic tasks from problems from your own software engineering work, along with the material needed to solve them.
• Give those tasks to frontier AI models and judge the results against the standard of your profession.
• Weigh paired model answers against each other and explain which one holds up.
• Write grading rubrics for your tasks and set out your reasoning in writing.
• Review and improve tasks written by other engineers.

Skills

Software engineering

Code review

Debugging

Testing

Git

Rubric writing

Education

Bachelor's degree or higher

Job description

Sovrano AI is seeking independent contractors for fully remote, project-based work evaluating AI coding agents. You will create tasks from your own software engineering experience, assign them to frontier AI models, and judge the results as a professional reviewer.

You set your own schedule and will be paid hourly, with $40–$125 per hour and a guaranteed minimum of 10 hours weekly. Pay is processed via PayPal and you are responsible for your own taxes as an independent contractor.