Software Engineering AI Response Evaluator

OpenTrain AI, Inc.; Northern (KY); Hybrid; USD 165,000 - 193,000; Part time

Benefits offered by this job

Remote, asynchronous work

Job summary

OpenTrain AI, Inc. in the United States seeks software engineers for remote, part-time contract work focused on creating realistic programming problems and reviewing AI responses.

You will assess technical accuracy and practicality while ensuring alignment with production engineering practices. Ideal candidates have at least one year of professional or research software engineering experience, a Bachelor's in CS or related field, and strong written communication to explain complex concepts

Qualifications

• Experience: At least one year of direct professional or research experience as a software engineer.
• Education: Bachelor's degree in Computer Science or a related technical field.
• Technical background: Depth in backend systems and distributed architecture, infrastructure and cloud platforms, DevOps, or systems programming and performance engineering.
• Language: English.

Responsibilities

• Create challenging software engineering problems and realistic scenarios based on system design, architecture, and implementation work.
• Evaluate AI-generated software engineering responses for technical accuracy and practical feasibility.
• Check whether model outputs reflect sound production engineering practices.
• Write clear, structured feedback that helps improve AI model performance.

Skills

Strong written communication

Attention to detail

Explain complex concepts clearly

Education

Bachelor's degree in Computer Science or related field

Job description

The Work

You will help improve AI systems by creating realistic software engineering problems and reviewing the answers they produce. Your feedback will focus on whether each answer is technically correct, practical, and consistent with production engineering practices.

This is ongoing, project-based contract work that you can complete remotely and asynchronously at your own pace.

• Create challenging software engineering problems and realistic scenarios based on system design, architecture, and implementation work.
• Evaluate AI-generated software engineering responses for technical accuracy and practical feasibility.
• Check whether model outputs reflect sound production engineering practices.
• Write clear, structured feedback that helps improve AI model performance.

What It Pays and Takes

This role is suited to software engineers with professional or research experience and strong written communication. You should be comfortable explaining complex technical ideas and working independently.

• Pay: $120 per hour.
• Time: 20 or more hours per week.
• Work type: Part-time, project-based contractor role.
• Location: United States only.
• Language: English.
• Experience: At least one year of direct professional or research experience as a software engineer.
• Education: Bachelor's degree in Computer Science or a related technical field.
• Technical background: Depth in backend systems and distributed architecture, infrastructure and cloud platforms, DevOps, or systems programming and performance engineering.
• Skills: Strong written communication, careful technical attention to detail, and the ability to explain complex software engineering concepts clearly.
• Work style: Able to work independently in a remote, asynchronous environment.
• Helpful background: Experience at a recognized top-tier technology company can strengthen your background for this work.

About AI Training Work

AI training work uses human judgment to help artificial intelligence learn from better examples and feedback. Software engineers are paid for this work because they can tell when an AI-generated answer is correct, practical, and suitable for real systems.