Remote AI Data Scientist — RLHF & Evaluation

OpenTrain AI; United States; On-site; USD 68,880 - 137,760; Part time

Job summary

OpenTrain AI is recruiting an AI Data Scientist contractor to evaluate AI-generated analytical reasoning, code, and model outputs, and to rate model responses for correctness and reasoning quality. The role focuses on model evaluation and training content in data science and ML.

This is a remote, part-time contractor engagement (20+ hours/week). Hourly pay ranges by country, with rates up to $100/hr depending on location, and open hiring in multiple countries including the United States.

Qualifications

• Bachelor’s degree or higher in a quantitative field.
• 5+ years of professional experience as a Data Scientist or related analytical role.
• Strong Python skills for data analysis and ML (pandas, NumPy, scikit-learn).
• Solid background in statistics, experimental design, and applied probability.
• Hands-on experience building, evaluating, and deploying machine learning models.
• Advanced SQL skills and comfort with large datasets.
• Minimum C1 English proficiency with the ability to write clear step-by-step explanations.
• Experience with data visualization and presenting findings to stakeholders.
• Prior experience reviewing AI-generated analytical content is a strong plus.
• Highly detail-oriented, systematic, and focused on reproducibility and careful evaluation of reasoning steps.

Responsibilities

• Review AI-generated analytical reasoning, code, and model outputs for accuracy, clarity, and methodological soundness.
• Write step-by-step reference solutions and clear quantitative explanations for complex problems.
• Fact-check quantitative claims and identify methodological or statistical errors.
• Rate and compare multiple AI responses by correctness, reasoning quality, and prompt adherence.
• Create detailed prompts and exemplar responses to support model learning across topics.
• Test models for inaccuracies, bias, and reliability across use cases, documenting reproducible findings.

Skills

5+ years experience

Python for ML

Statistics

Experiment design

English proficiency

Data visualization

Reproducibility

Education

Bachelor’s degree or higher in Data Science / CS / Statistics / Math

Tools

Python

pandas

NumPy

scikit-learn

SQL

Job description

OpenTrain AI is recruiting an AI Data Scientist contractor to evaluate AI-generated analytical reasoning, code, and model outputs, and to rate model responses for correctness and reasoning quality. The role focuses on model evaluation and training content in data science and ML.

This is a remote, part-time contractor engagement (20+ hours/week). Hourly pay ranges by country, with rates up to $100/hr depending on location, and open hiring in multiple countries including the United States.