Senior Software Engineer, LLM Code Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 55,000 - 110,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Senior Software Engineer to help build and evaluate code for AI systems. This remote, part-time role focuses on analyzing AI-generated code, developing verification tools, and creating robust evaluation pipelines to ensure high-quality software delivery.

You will review code for efficiency and reliability, design automated checks, and collaborate with cross-functional teams to improve AI coding solutions and deployment practices.

Qualifications

• Strong full-stack development skills and production deployment experience.
• Deep knowledge of software architecture, design, development, debugging, and code review.
• Experience with multiple programming languages.
• Fluent in English with strong communication skills for clear explanations of evaluations.
• Experience designing automated verification methods is helpful.

Responsibilities

• Review AI-generated code for efficiency, scalability, reliability, and performance.
• Build verification agents and other checks that find recurring errors and validate solutions automatically.
• Assess model abilities across prototyping, architecture and API design, production implementation, launch, experimentation, monitoring, and ongoing maintenance.
• Work with cross-functional teams to improve AI coding solutions.

Skills

Full-stack development

Software architecture

Multiple languages

Job description

Senior Software Engineer, LLM Code Evaluation

Coding & Software

Remote

Contract, part-time

Engagement

Remote

Location

Posted

Open to applicants in

United States

The work

You will help create training and benchmark datasets that assess how well AI models handle software engineering. The work involves reviewing and refining code, writing precise solutions, and explaining your judgments clearly.

• Review AI-generated code for efficiency, scalability, reliability, and performance.
• Build verification agents and other checks that find recurring errors and validate solutions automatically.
• Assess model abilities across prototyping, architecture and API design, production implementation, launch, experimentation, monitoring, and ongoing maintenance.
• Work with cross-functional teams to improve AI coding solutions.

What you need

• Strong full-stack application development skills and production deployment experience.
• Deep knowledge of software architecture, design, development, debugging, and code review.
• Working ability in multiple programming languages named above.
• Fluent in English, with strong spoken and written communication skills for clear, structured evaluation explanations.
• Experience designing automated verification methods and reasoning across the software development lifecycle is helpful.

Schedule and location

• Contractor engagement with flexible scheduling and a requirement of 20 or more hours per week.
• Open to candidates based in the United States.
• The initial engagement is one month, with possible extensions based on performance and fit.

About AI training work

AI training work involves people preparing and reviewing examples that help AI models learn. In this role, software engineering expertise helps assess code and explain whether AI-generated solutions work well.