Remote AI Model Auditor & Evaluator (LLMs, RLHF)

prolificacademicltd; United States; Remote; USD 83,000 - 110,000; Full time

Benefits offered by this job

Fully remote

Flexible scheduling

Short, focused sessions

Task-based engagements

Job summary

PROLIFIC ACADEMIC LTD is offering a paid, remote, task-based research opportunity for experienced ML engineers to shape the next generation of large language models. You will audit, evaluate, and refine AI outputs, requiring deep technical judgment and careful attention to model behavior.

Engagements are flexible and intermittent, designed to fit alongside other professional commitments, with sessions typically hour-long and several shorter tasks.

Qualifications

• Bachelor's, Master's, or PhD in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field with a machine learning focus.
• Professional experience building, deploying, or fine-tuning ML models in production.
• Strong command of neural network architectures such as Transformers, CNNs, and RNNs, plus modern optimization techniques.
• Hands-on work with prompt engineering, RLHF, or retrieval-augmented generation pipelines.
• Sharp analytical eye for spotting biased outputs, training data contamination, and faulty reasoning.

Responsibilities

• Review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit ML code, training scripts, and evaluation notebooks for correctness and efficiency.
• Provide structured human feedback used to align models through reinforcement learning from human feedback workflows.
• Critically analyze chain-of-thought reasoning and surface logical failures or hallucinations.
• Run comparative benchmarks across model outputs using defined technical taxonomies and metrics.

Skills

ML model development

Transformers

Prompt engineering

RLHF

Bias detection

Education

Bachelor's degree in CS/AI/Robotics or related

Master's degree

PhD

Tools

PyTorch

TensorFlow

Keras

Hugging Face Transformers

Weights & Biases

LangChain

Pinecone

Job description

PROLIFIC ACADEMIC LTD is offering a paid, remote, task-based research opportunity for experienced ML engineers to shape the next generation of large language models. You will audit, evaluate, and refine AI outputs, requiring deep technical judgment and careful attention to model behavior.

Engagements are flexible and intermittent, designed to fit alongside other professional commitments, with sessions typically hour-long and several shorter tasks.