Remote Senior AI/ML Auditor & Evaluator

prolificacademicltd; United States; Remote; USD 66,000 - 110,000; Full time

Benefits offered by this job

Remote work

Flexible scheduling

Job summary

prolificacademicltd is seeking senior AI/ML engineers for a remote, task-based engagement focused on training and evaluating large language models. You will provide expert critique of model behavior, code, and reasoning to improve performance and safety.

Responsibilities include evaluating explanations, auditing code and training loops, giving RLHF feedback, analyzing chain-of-thought errors, and benchmarking outputs against technical criteria, with flexible scheduling and compensation up to $80

Qualifications

• BS/MS/PhD in Computer Science, AI, Robotics, or related quantitative ML discipline
• Production experience with ML models
• Understanding of neural architectures (Transformers, CNNs, RNNs) and optimization methods
• Hands-on with prompt engineering, RLHF, or retrieval-augmented generation workflows
• Ability to audit complex model logic and detect data contamination

Responsibilities

• Evaluate AI-generated explanations for technical accuracy
• Audit ML code, notebooks, training loops, and preprocessing pipelines
• Provide expert feedback to strengthen RLHF for safety and alignment
• Analyze chain-of-thought reasoning to identify failures
• Run structured benchmarking across model outputs against technical criteria

Skills

Deep learning expertise

Transformer models

Python programming

Prompt engineering

RLHF

Model evaluation

Education

PhD in CS/AI

Tools

PyTorch

TensorFlow/Keras

Hugging Face Transformers

NumPy/Pandas

Weaviate

Job description

prolificacademicltd is seeking senior AI/ML engineers for a remote, task-based engagement focused on training and evaluating large language models. You will provide expert critique of model behavior, code, and reasoning to improve performance and safety.

Responsibilities include evaluating explanations, auditing code and training loops, giving RLHF feedback, analyzing chain-of-thought errors, and benchmarking outputs against technical criteria, with flexible scheduling and compensation up to $80