Remote AI Model Quality Auditor (Paid Tasks)

prolificacademicltd; United States; Remote; USD 91,000 - 129,000; Full time

Benefits offered by this job

Flexible remote work

Competitive hourly pay

Job summary

Prolific Academic Ltd. offers remote, paid, task-based work evaluating large language models. You will review explanations, code, and model outputs to identify errors and improve reliability. A short skills assessment precedes invitation to participate.

You will assess architectures, training loops, data prep, and results, providing expert feedback to support RLHF and model alignment. Flexible hours, contract-based pay, and fast onboarding are offered.

Qualifications

• Bachelor's, master's, or doctoral degree in computer science, AI, robotics, or a related quantitative field focused on machine learning.
• Professional experience building, deploying, or fine-tuning machine learning models in production.
• Strong understanding of neural network architectures such as Transformers, CNNs, and RNNs, as well as optimization methods.
• Hands-on experience with prompt engineering, RLHF, or retrieval-augmented generation workflows.
• Advanced Python skills and expertise with PyTorch or TensorFlow/Keras; familiarity with common data and model libraries is relevant.
• Careful analytical judgment when evaluating technical claims, code, and model outputs.

Responsibilities

• Check AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Review machine learning code and notebooks, including training loops, data preparation, and evaluation work, for correctness and efficiency.
• Provide expert feedback that supports reinforcement learning from human feedback and model alignment.
• Assess model reasoning on complex technical prompts and identify where the logic fails.
• Compare model outputs using defined technical categories and performance measures.
• Review content for hallucinations, bias, logical errors, or possible training data contamination.

Skills

ML production experience

Transformers/CNNs/RNNs

Prompt engineering / RLHF / RAG

Python + PyTorch or TensorFlow/Keras

Analytical evaluation skills

Education

Bachelor's/Master's/Doctoral in CS/AI/Robotics or ML

Tools

PyTorch

TensorFlow/Keras

Cloud ML platforms / MLOps

Vector databases for RAG

Job description

Prolific Academic Ltd. offers remote, paid, task-based work evaluating large language models. You will review explanations, code, and model outputs to identify errors and improve reliability. A short skills assessment precedes invitation to participate.

You will assess architectures, training loops, data prep, and results, providing expert feedback to support RLHF and model alignment. Flexible hours, contract-based pay, and fast onboarding are offered.