AI Engineer/ML Engineer - Senior Developers - AI Training - San Francisco, US

Prolific; San Francisco (CA); On-site; USD 90,921 - 129,494; Full time

Job summary

Prolific is seeking an AI & Machine Learning Engineer in San Francisco to evaluate and refine AI models. The role involves auditing ML code, providing human feedback on AI frameworks, and analyzing model reasoning. Candidates should have a relevant degree and experience in deploying ML models. The position offers up to $80 per hour with flexible work-from-home hours. This is an excellent opportunity for those passionate about advancing AI and Machine Learning technologies.

Qualifications

• Education in Computer Science, AI, Robotics, or a related field focusing on Machine Learning.
• Experience deploying or fine-tuning ML models in production.
• Mastery of neural network architectures and optimization techniques.

Responsibilities

• Evaluate LLM architecture logic for technical accuracy.
• Audit code and notebooks for efficiency and correctness.
• Refine RLHF frameworks with high-quality human feedback.
• Analyze AI model reasoning and identify breakdowns.
• Benchmark performance between different model outputs.

Skills

AI model evaluation

Code auditing

RLHF frameworks

Deep Learning understanding

Analytical critique

Education

BS, MS, or PhD in Computer Science, AI, or related field

Job description

Overview

AI & Machine Learning Engineer - AI Training at Prolific. Prolific is building the biggest pool of quality human data in the world. Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid study participants with a wide variety of experiences, knowledge, and skills.

Responsibilities

• Evaluate LLM architecture logic: review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit code & notebooks: validate ML-specific code (training loops, data preprocessing scripts, model evaluations) for efficiency and correctness.
• Refine RLHF frameworks: provide high-quality human feedback to align models with human intent, safety, and usefulness.
• Analyze model reasoning: assess how an AI model navigates complex chain-of-thought prompts and identify where reasoning breaks down.
• Benchmark performance: conduct comparative testing between different model outputs based on technical taxonomies and performance metrics.

Qualifications

• Education: a BS, MS, or PhD in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field with a focus on Machine Learning.
• Professional Experience: experience building, deploying, or fine-tuning ML models in a production environment.
• Deep Learning Mastery: professional-level understanding of neural network architectures (Transformers, CNNs, RNNs) and optimization techniques.
• LLM Specialization: hands-on experience with Prompt Engineering, RLHF, or RAG workflows.
• Technical Rigor: ability to audit complex model logic, identify training data contamination, and evaluate mathematical proofs behind ML algorithms.
• Analytical Critique: high attention to detail in spotting hallucinations, biased outputs, or logical failures in AI-generated technical content.

Compensation & Why Join

• Pay rate up to $80 per hour; many tasks require one hour of uninterrupted work, though many are shorter.
• As a participant, you will be invited to join Prolific after a quick assessment and will be paid to train and evaluate AI models.
• Flexible hours and the ability to work from home.