Senior AI Engineer - LLM Training & Benchmarking (Remote)

prolificacademicltd; United States; Remote; USD 66,000 - 110,000; Full time

Job summary

Prolific Academic Ltd is seeking senior AI and Machine Learning Engineers to train, critique, and benchmark large language models on a paid, task-based basis. Work is fully remote with flexible scheduling, letting you choose tasks that fit within roughly an hour of focused effort.

You will audit architectures, code, and RLHF signals, compare outputs against structured taxonomies, and provide precise feedback to improve safety and usefulness.

Qualifications

• BS, MS, or PhD in Computer Science, AI, Robotics or related quantitative field with ML concentration.
• Hands-on production experience building, deploying, or fine-tuning ML models.
• Knowledge of neural network architectures such as Transformers, CNNs, RNNs, and modern optimization techniques.
• Practical experience with prompt engineering, RLHF, or retrieval-augmented generation workflows.
• Ability to audit intricate model logic, detect data contamination, and reason about ML proofs.

Responsibilities

• Scrutinize AI-generated descriptions of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit ML code and notebooks for efficiency and correctness.
• Supply human feedback signals that drive RLHF alignment and model safety.
• Analyze how models navigate chain-of-thought prompts and identify failure points.
• Benchmark model outputs against structured technical taxonomies and quantitative criteria.

Skills

Transformers

RNNs

CNNs

prompt engineering

RLHF

Model auditing

Education

BS/MS/PhD in CS/AI/Robotics with ML concentration

Tools

PyTorch

TensorFlow/Keras

AWS SageMaker

Google Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

Prolific Academic Ltd is seeking senior AI and Machine Learning Engineers to train, critique, and benchmark large language models on a paid, task-based basis. Work is fully remote with flexible scheduling, letting you choose tasks that fit within roughly an hour of focused effort.

You will audit architectures, code, and RLHF signals, compare outputs against structured taxonomies, and provide precise feedback to improve safety and usefulness.