Senior AI/ML Engineer - LLM Training & Evaluation

Prolific Academic Ltd; United States; Remote; USD 91,000 - 129,000; Full time

Benefits offered by this job

Flexible hours

Work from home

Paid AI training tasks

Job summary

Prolific Academic Ltd seeks an AI & Machine Learning Engineer to train and evaluate next-generation AI models within its Expert Network. You will be paid to complete paid tasks by researchers seeking high-level ML expertise, with the possibility to join Prolific as a participant after assessment.

Tasks vary and may require focused one-hour work blocks. The role emphasizes deep learning mastery, LLM specialization, and careful evaluation of model outputs, with competitive pay and flexible

Qualifications

• BS, MS, or PhD in CS/AI with ML focus.
• Experience building, deploying, or fine-tuning ML models in production.
• Deep learning mastery: transformers, CNNs, RNNs, and optimization techniques.
• LLM specialization: Prompt Engineering, RLHF, or RAG workflows.
• Technical rigor: audit model logic, data contamination, and proofs behind ML algorithms.
• Analytical critique: detect hallucinations, biases, or logical failures in AI content.

Responsibilities

• Evaluate LLM Architecture Logic: review AI-generated explanations for technical accuracy.
• Audit Code & Notebooks: validate ML-specific code for efficiency and correctness.
• Refine RLHF Frameworks: provide high-quality human feedback to align models with intent, safety and usefulness.
• Analyze Model Reasoning: assess chain-of-thought prompts and diagnose failures.
• Benchmark Performance: conduct comparative testing based on taxonomies and metrics.

Skills

Education: BS/MS/PhD

ML deployment

DL Mastery

LLM specialization

Technical rigor

Analytical critique

Education

BS/MS/PhD in CS/AI

Tools

PyTorch

TensorFlow/Keras

HuggingFace Transformers

AWS SageMaker

Vertex AI

Weights & Biases

LangChain

Pinecone

Job description

Prolific Academic Ltd seeks an AI & Machine Learning Engineer to train and evaluate next-generation AI models within its Expert Network. You will be paid to complete paid tasks by researchers seeking high-level ML expertise, with the possibility to join Prolific as a participant after assessment.

Tasks vary and may require focused one-hour work blocks. The role emphasizes deep learning mastery, LLM specialization, and careful evaluation of model outputs, with competitive pay and flexible