Remote AI/ML Model Auditor & Evaluator

prolificacademicltd; United States; Remote; USD 83,000 - 110,000; Full time

Job summary

prolificacademic ltd is seeking experienced AI/ML engineers to join a paid expert network. Work is remote and task-based, focusing on auditing, critiquing, and improving model behavior rather than shipping production systems.

Responsibilities include reviewing explanations of architectures, auditing training loops and evaluation pipelines, and supplying high-quality feedback to align models with human intent via RL workflows. Strong emphasis on prompt engineering and model safety.

Qualifications

• BS, MS, or PhD in CS/AI/Robotics or related quantitative field focusing on ML.
• Hands-on experience building, deploying, or fine-tuning ML models in production.
• Knowledge of neural networks (Transformers/CNNs/RNNs) and optimization techniques.
• Experience with prompt engineering, RLHF, or retrieval-augmented generation workflows.
• Ability to audit complex model logic and assess training data contamination.
• Strong attention to detail to catch hallucinations and biases in outputs.

Responsibilities

• Review explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit ML code, training loops, preprocessing scripts, and evaluation pipelines for correctness.
• Provide high-quality human feedback to align models via RL workflows.
• Analyze how models reason through chain-of-thought prompts and identify logic failures.
• Run benchmarks between model outputs using technical taxonomies and metrics.

Skills

ML model development

ML deployment

Prompt engineering

RLHF

RAG evaluation

Model auditing

Education

BS in CS/AI/Robotics

MS in ML field

PhD in AI

Tools

PyTorch

TensorFlow

Keras

NumPy

Pandas

Scikit-learn

Hugging Face Transformers

SageMaker

Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

prolificacademic ltd is seeking experienced AI/ML engineers to join a paid expert network. Work is remote and task-based, focusing on auditing, critiquing, and improving model behavior rather than shipping production systems.

Responsibilities include reviewing explanations of architectures, auditing training loops and evaluation pipelines, and supplying high-quality feedback to align models with human intent via RL workflows. Strong emphasis on prompt engineering and model safety.