Senior Python Engineer, LLM Evaluation (US-Based)

OpenTrain AI, Inc.; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

OpenTrain AI, Inc. seeks a Senior Python Engineer for LLM evaluation. This US-based contractor role is remote, part-time, and requires 20+ hours per week with at least three years of software engineering experience.

You will build Python tools and evaluate AI-generated code for correctness, efficiency, and reliability. You will develop solutions for backend services, data pipelines, and ML infrastructure, reviewing code quality and contributing to automated evaluation systems.

Qualifications

• Deep Python knowledge and production frameworks.
• Experience building full-stack apps and scalable deployments.
• Strong software architecture, design, debugging and code quality review.
• Ability to write clear, structured evaluation rationales.
• U.S.-based candidates preferred; contract at 20+ hours/week.

Responsibilities

• Prepare code examples and write precise solutions to software engineering tasks.
• Review and improve AI-generated code for correctness, efficiency, scalability, and reliability.
• Build Python agents and automated tools to find error patterns and assess code quality.
• Design ways to automatically check solutions to programming problems.
• Assess model performance across prototyping, architecture and API design, implementation, launch, experimentation, monitoring, and maintenance.
• Work with other teams to improve AI coding systems against performance benchmarks.

Skills

Python

Production frameworks

Full-stack development

Software architecture

Code quality review

Evaluation reasoning

Job description

Senior Python Engineer, LLM Evaluation (US-Based)

Build Python tools and evaluate AI-generated code for correctness, efficiency, and reliability. This US-based contractor role requires 20+ hours a week and at least three years of software engineering experience.

Remote

Contract, part-time

The work

Create and improve software engineering examples and solutions used to train and assess large language models. You will focus on Python code for backend services, data pipelines, and machine learning infrastructure, with work across other programming languages too.

• Prepare code examples and write precise solutions to software engineering tasks.
• Review and improve AI-generated code for correctness, efficiency, scalability, and reliability.
• Build Python agents and automated tools to find error patterns and assess code quality.
• Design ways to automatically check solutions to programming problems.
• Assess model performance across prototyping, architecture and API design, implementation, launch, experimentation, monitoring, and maintenance.
• Work with other teams to improve AI coding systems against performance benchmarks.

What you need and engagement details

• At least three years of software engineering experience.
• Deep Python knowledge, including production frameworks, tools, and best practices.
• Experience building full-stack applications and deploying software that can scale.
• Strong skills in software architecture, design, development, debugging, and code quality review.
• Ability to write and explain clear, structured evaluation rationales.
• Open to candidates based in the United States.
• Plan to contribute at least 20 hours per week.
• Contractor engagement with an initial one-month duration; extensions may be available based on performance and fit.

About AI training work

AI training work is the human work of preparing examples and reviewing model responses so AI systems can learn and be evaluated. This role draws on software engineering experience to create code examples and judge whether generated programs work well.

Create and solve biology research problems, build Python analyses and models, and review scientific code for AI evaluation. Remote, flexible work averages about 15 hours weekly and pays $60 to $120 per hour equivalent, paid per task that meets project specifications.

Research vulnerabilities in private code repositories and turn findings into challenging AI security tasks. The role pays $100 per hour and requires at least 20 hours a week.

Create coding challenges and assess AI-generated software solutions for technical accuracy and good engineering practice. The role pays $40 to $60 per hour and requires at least 20 hours per week.

Build Python tools to verify code and assess AI-generated solutions across backend, data, machine learning infrastructure, and full-stack projects. This remote US contractor role calls for 20+ hours per week.

Evaluate AI-generated code, create solutions and Python checks, and assess software quality across several languages. This entry-level US-based contractor role requires 20+ hours a week and at least three years of software engineering experience.