Remote Senior Python Engineer - LLM Evaluation & QA

OpenTrain AI, Inc.; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

OpenTrain AI, Inc. seeks a Senior Python Engineer for LLM evaluation. This US-based contractor role is remote, part-time, and requires 20+ hours per week with at least three years of software engineering experience.

You will build Python tools and evaluate AI-generated code for correctness, efficiency, and reliability. You will develop solutions for backend services, data pipelines, and ML infrastructure, reviewing code quality and contributing to automated evaluation systems.

Qualifications

• Deep Python knowledge and production frameworks.
• Experience building full-stack apps and scalable deployments.
• Strong software architecture, design, debugging and code quality review.
• Ability to write clear, structured evaluation rationales.
• U.S.-based candidates preferred; contract at 20+ hours/week.

Responsibilities

• Prepare code examples and write precise solutions to software engineering tasks.
• Review and improve AI-generated code for correctness, efficiency, scalability, and reliability.
• Build Python agents and automated tools to find error patterns and assess code quality.
• Design ways to automatically check solutions to programming problems.
• Assess model performance across prototyping, architecture and API design, implementation, launch, experimentation, monitoring, and maintenance.
• Work with other teams to improve AI coding systems against performance benchmarks.

Skills

Python

Production frameworks

Full-stack development

Software architecture

Code quality review

Evaluation reasoning

Job description

OpenTrain AI, Inc. seeks a Senior Python Engineer for LLM evaluation. This US-based contractor role is remote, part-time, and requires 20+ hours per week with at least three years of software engineering experience.

You will build Python tools and evaluate AI-generated code for correctness, efficiency, and reliability. You will develop solutions for backend services, data pipelines, and ML infrastructure, reviewing code quality and contributing to automated evaluation systems.