Senior AI Training Engineer - LLM Evaluation & RLHF

Prolific Academic Ltd; United States; Remote; USD 83,000 - 138,000; Full time

Benefits offered by this job

Flexible hours

Work from home

Competitive pay

Paid tasks

Job summary

Prolific is seeking AI and Machine Learning Engineers to join our Expert Network to train and evaluate the next generation of LLMs with deep technical expertise. You’ll complete a brief 10–15 minute test; if successful, you’ll be invited to join Prolific as a participant to train and evaluate powerful AI models.

Researchers looking for your skills tend to pay up to $80 per hour, and many tasks require one hour of uninterrupted work, with flexibility and work-from-home options.

Qualifications

• BS/MS/PhD in CS/AI/Robotics with ML focus.
• Experience building, deploying, or fine-tuning ML models in production.
• Deep learning mastery of transformers, CNNs, RNNs.
• Hands-on experience with RLHF or RAG workflows.
• Ability to audit complex model logic and data contamination.

Responsibilities

• Evaluate LLM architecture logic for technical accuracy.
• Audit ML code/notebooks for efficiency and correctness.
• Refine RLHF frameworks with high-quality human feedback.
• Analyze model reasoning and CoT prompts for failures.
• Benchmark model outputs across taxonomies and metrics.

Skills

ML model development

Production ML deployment

Deep learning architectures

Prompt engineering

RLHF / RAG workflows

Code & notebook audit

Data contamination auditing

Statistical reasoning

Education

BS/MS/PhD in CS/AI/Robotics

Tools

PyTorch

TensorFlow/Keras

Hugging Face Transformers

SageMaker

Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

Prolific is seeking AI and Machine Learning Engineers to join our Expert Network to train and evaluate the next generation of LLMs with deep technical expertise. You’ll complete a brief 10–15 minute test; if successful, you’ll be invited to join Prolific as a participant to train and evaluate powerful AI models.

Researchers looking for your skills tend to pay up to $80 per hour, and many tasks require one hour of uninterrupted work, with flexibility and work-from-home options.