Senior AI/ML Engineer — LLM Training & Evaluation (Remote)

Prolific; United States; Remote; USD 91,000 - 129,000; Full time

Job summary

Prolific is seeking an AI & Machine Learning Engineer to join its Expert Network, focusing on training and evaluating next‑gen LLMs. You will work on challenging tasks that require deep technical expertise, including test assessments and paid tasks for alignment with human intent and safety.

You’ll be paid for one‑hour tasks, with flexible hours and remote work options. The role emphasizes prompt engineering, RLHF, and RAG workflows, with opportunities to influence model behavior, data quality,

Qualifications

• BS/MS/PhD in Computer Science, Artificial Intelligence, Robotics or related quantitative field with ML focus.
• Experience building, deploying, or fine-tuning ML models in production.
• Professional-level understanding of neural networks (Transformers, CNNs, RNNs) and optimization techniques.
• Hands-on prompt engineering, RLHF or RAG workflows.
• Ability to audit complex model logic and data contamination, and evaluate ML proofs.

Responsibilities

• Review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Validate ML-specific code for efficiency and correctness (training loops, preprocessing, evaluations).
• Provide high-quality human feedback to align models with human intent, safety and usefulness.
• Critically assess model reasoning in chain-of-thought prompts and identify breakdowns.
• Conduct benchmark tests comparing model outputs using defined metrics and taxonomies.

Skills

Deep learning

Model evaluation

Analytical critique

Attention to detail

Education

BS/MS/PhD in CS/AI/Robotics

Tools

PyTorch

TensorFlow/Keras

NumPy

Pandas

Scikit-learn

Hugging Face Transformers

AWS SageMaker

Google Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

Prolific is seeking an AI & Machine Learning Engineer to join its Expert Network, focusing on training and evaluating next‑gen LLMs. You will work on challenging tasks that require deep technical expertise, including test assessments and paid tasks for alignment with human intent and safety.

You’ll be paid for one‑hour tasks, with flexible hours and remote work options. The role emphasizes prompt engineering, RLHF, and RAG workflows, with opportunities to influence model behavior, data quality,