Senior AI/ML Evaluator (Remote, Per-Study)

prolificacademicltd; United States; Remote; USD 83,000 - 110,000; Part time

Benefits offered by this job

Fully remote

Flexible hours

Short onboarding

Job summary

prolificacademicltd seeks senior AI/ML engineers to join an expert network contributing to large language model training and evaluation. Roles are task-based, with participants paid per study for applying deep technical judgment to model outputs and ML artifacts.

Responsibilities include auditing training code, evaluating explanations, and providing structured feedback for RLHF pipelines. Fully remote, hourly pay up to $80, with brief onboarding and flexible hours.

Qualifications

• BS, MS, or PhD in Computer Science, AI, Robotics or related quantitative field with ML focus.
• Professional experience building, deploying, or fine-tuning ML models in production environments.
• Working knowledge of neural architectures (Transformers, CNNs, RNNs) and optimization techniques.
• Hands-on with prompt engineering, RLHF, or retrieval-augmented generation workflows.
• Ability to audit complex model logic and identify data contamination.

Responsibilities

• Assess AI-generated explanations of neural architectures, loss functions and backpropagation for technical correctness.
• Validate ML code such as training loops, preprocessing scripts and evaluation notebooks for efficiency and correctness.
• Provide structured human feedback powering RLHF pipelines to align models with safety and usefulness criteria.
• Trace how models handle chain-of-thought prompts and identify breakdowns in reasoning.
• Conduct benchmarking of model outputs against technical taxonomies and metrics.

Skills

ML in production

Prompt engineering

RLHF workflows

Model evaluation

Error analysis

Education

PhD in ML

Tools

PyTorch

TensorFlow

Keras

Hugging Face

AWS SageMaker

LangChain

Pinecone

Milvus

Weaviate

Job description

prolificacademicltd seeks senior AI/ML engineers to join an expert network contributing to large language model training and evaluation. Roles are task-based, with participants paid per study for applying deep technical judgment to model outputs and ML artifacts.

Responsibilities include auditing training code, evaluating explanations, and providing structured feedback for RLHF pipelines. Fully remote, hourly pay up to $80, with brief onboarding and flexible hours.