Senior Software Engineer, LLM Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 83,000 - 124,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Senior Software Engineer for LLM Evaluation. This remote, US-based contractor role offers a flexible schedule of 20+ hours per week.

The role focuses on Python across backend services, data pipelines, ML infrastructure, and full-stack development. You will build Python agents and automated tools to verify code and assess AI-generated solutions, evaluating correctness, efficiency, and scalability while collaborating with cross-functional teams to improve coding

Qualifications

• Minimum three years in software engineering.
• Strong Python skills including production frameworks and development practices.
• Experience building full-stack applications and deploying scalable software.
• Good knowledge of software architecture, development, debugging and code review.

Responsibilities

• Create and improve code examples, develop solutions, and review AI-generated code.
• Use software engineering judgment to assess correctness, efficiency, scalability, reliability, and overall quality.
• Work primarily in Python across backend services, data pipelines, ML infrastructure, and full-stack development.
• Build Python agents and automated tools to verify code and assess AI-generated solutions.

Skills

Python

Full-stack development

Software architecture

Code review

Debugging

Production frameworks

Job description

Senior Software Engineer, LLM Evaluation

Assess AI-generated code and build Python tools that check software solutions across backend, data, ML infrastructure, and full-stack work. This US-based contractor role offers a flexible schedule of 20 or more hours per week.

Remote

Contract, part-time

Engagement

Remote

Location

Posted

Open to applicants in

United States

The work

Create and improve code examples, develop solutions, and review AI-generated code. You will use software engineering judgment to assess correctness, efficiency, scalability, reliability, and overall quality.

• Work primarily in Python across backend services, data pipelines, machine learning infrastructure, and full-stack development.
• Build Python agents and automated tools to find error patterns, check solutions, and assess code quality.
• Evaluate model skills in prototyping, architecture, API design, production implementation, monitoring, and ongoing maintenance.
• Work with cross-functional teams to improve coding systems against industry benchmarks.

Requirements and engagement

• Location: Open to candidates based in the United States.
• Schedule: Flexible, with a commitment of at least 20 hours per week.
• Engagement: Part-time contractor role; initial duration is one month, with possible extensions based on performance and fit.
• Experience: At least three years in software engineering.
• Strong Python skills, including production frameworks, tools, and development practices.
• Experience building full-stack applications and deploying scalable software.
• Strong knowledge of software architecture, development, debugging, and code review.
• Clear spoken and written communication for explaining evaluation decisions.
• Experience with advanced AI systems or organizations developing intelligent software is helpful. Strong computer science foundations are welcome.

About AI training work

AI training work is the human work behind AI systems: people prepare examples and review model outputs so the systems can learn. Software engineers are needed to judge code quality and build reliable ways to test AI-generated solutions.

Requirements

Job details

Location

100% remote

United States

Languages

Engagement

Contract, part-time

Coding & Software

Data type

Computer Code Programming

Computer Programming Coding, Evaluation Rating

Build Python tools to verify code and assess AI-generated solutions across backend, data, machine learning infrastructure, and full-stack projects. This remote US contractor role calls for 20+ hours per week.

Evaluate AI-generated code, create solutions and Python checks, and assess software quality across several languages. This entry-level US-based contractor role requires 20+ hours a week and at least three years of software engineering experience.