Remote AI Engineer - Model QA & RLHF Evaluation

prolificacademicltd; United States; Remote; USD 83,000 - 110,000; Full time

Benefits offered by this job

Remote work

Flexible hours

Skills assessment onboarding

Job summary

Prolific Academic Ltd seeks experienced AI/ML engineers to join a remote expert network. Engage in reviewing, auditing, and refining outputs for next-generation language models, applying deep ML knowledge from home with flexible hours.

Responsibilities include auditing training loops, data preprocessing, and evaluation pipelines; providing RLHF feedback; and benchmarking outputs against technical metrics. Portland-area preference noted.

Qualifications

• Bachelor's/Master/PhD in Computer Science, AI, Robotics or related quantitative field with ML focus
• Hands-on experience building, deploying, or fine-tuning ML models in production
• Knowledge of neural network architectures (Transformers, CNNs, RNNs) and modern optimization techniques
• Experience with prompt engineering, RLHF, or retrieval-augmented generation (RAG) workflows
• Strong analytical rigor with ability to audit complex model logic and detect data contamination
• Attention to detail in identifying hallucinations, biases and logical failures in AI content

Responsibilities

• Review AI explanations of model architectures, loss functions, and backpropagation for technical accuracy
• Audit ML code and notebooks, including training loops, data preprocessing scripts, and evaluation pipelines
• Provide high-quality human feedback used in RLHF workflows to align model behavior with human intent
• Critically evaluate chain-of-thought reasoning and identify where model logic breaks down
• Conduct benchmarking of model outputs against defined technical taxonomies and metrics

Skills

ML model training

RLHF

Prompt engineering

Model evaluation

Education

BS/MS/PhD in CS/AI/Robotics

Tools

PyTorch

TensorFlow

Hugging Face

LangChain

Weights & Biases

Vertex AI

SageMaker

Pinecone

Weaviate

Job description

Prolific Academic Ltd seeks experienced AI/ML engineers to join a remote expert network. Engage in reviewing, auditing, and refining outputs for next-generation language models, applying deep ML knowledge from home with flexible hours.

Responsibilities include auditing training loops, data preprocessing, and evaluation pipelines; providing RLHF feedback; and benchmarking outputs against technical metrics. Portland-area preference noted.