LLM Training & Evaluation Engineer (Remote)

Prolific; United States; Remote; USD 130,000 - 190,000; Full time

Job summary

Prolific is seeking AI & Machine Learning Engineers to join our Expert Network, training and evaluating next‑gen LLMs. You’ll tackle architecture reviews, code audits, and RLHF/RAG workflows while earning competitive pay for one‑hour tasks or shorter sessions, with the flexibility to work from home.

Successful candidates will have advanced ML degrees or equivalent experience, hands‑on model deployment, and a solid track record in deep learning with Transformers and AI safety considerations.

Qualifications

• BS/MS/PhD in CS/AI/Robotics with ML focus.
• Experience building, deploying, or fine-tuning ML models in production.
• Deep Learning mastery with Transformers/CNNs/RNNs.
• LLM specialization: Prompt Engineering, RLHF, or RAG workflows.
• Technical rigor: audit complex model logic and data contamination.
• Analytical critique: detect hallucinations and biased outputs.

Responsibilities

• Evaluate LLM Architecture Logic for technical accuracy.
• Audit ML code/notebooks for efficiency and correctness.
• Refine RLHF frameworks with high-quality human feedback.
• Analyze model reasoning and identify CoT failure points.
• Benchmark performance across model outputs with metrics.

Skills

ML model development

Model evaluation

RLHF experience

RAG workflows

Audit complex models

Error analysis

Education

BS/MS/PhD in CS/AI/Robotics with ML focus

Tools

PyTorch

TensorFlow/Keras

Python

Hugging Face Transformers

AWS SageMaker

Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

Prolific is seeking AI & Machine Learning Engineers to join our Expert Network, training and evaluating next‑gen LLMs. You’ll tackle architecture reviews, code audits, and RLHF/RAG workflows while earning competitive pay for one‑hour tasks or shorter sessions, with the flexibility to work from home.

Successful candidates will have advanced ML degrees or equivalent experience, hands‑on model deployment, and a solid track record in deep learning with Transformers and AI safety considerations.