Remote AI Model Evaluator & Benchmark Scientist

OpenTrain AI, Inc.; United States; Remote; USD 34,000 - 55,000; Part time

Job summary

OpenTrain AI, Inc. is seeking an entry-level contractor to join our AI training program. You will design, implement, and evaluate Python-based data analysis workflows, and create datasets and clear explanations of results.

You will collaborate with researchers and annotators on supervised fine-tuning, RLHF, and reward-model refinement. Remote work, global applicants, with 20–40 hours weekly and overlap with Pacific Time.

Qualifications

• Bachelor's or Master's degree in engineering or computer science (or equivalent).
• Entry-level experience in Python and data analysis.
• Strong problem-solving ability and sound business judgment.

Responsibilities

• Design, develop, and maintain Python code for training and optimizing AI models.
• Benchmark model performance and analyze evaluation results.
• Evaluate and rank model responses to user queries using set criteria.
• Write clear explanations and rationales for evaluation decisions.
• Create and maintain datasets for supervised fine-tuning.
• Create and refine responses for clarity, relevance, and technical accuracy.
• Review code and documentation and provide constructive feedback.
• Use public datasets from sources such as Kaggle, the United Nations, and the US government.

Skills

Python

Data analysis

Problem-solving

Business judgment

Communication

Education

Bachelor's or Master’s in Engineering/CS

Tools

Jupyter notebooks

Job description

OpenTrain AI, Inc. is seeking an entry-level contractor to join our AI training program. You will design, implement, and evaluate Python-based data analysis workflows, and create datasets and clear explanations of results.

You will collaborate with researchers and annotators on supervised fine-tuning, RLHF, and reward-model refinement. Remote work, global applicants, with 20–40 hours weekly and overlap with Pacific Time.