Senior AI Auditor & RLHF Specialist (Remote)

prolificacademicltd; United States; Remote; USD 91,000 - 129,000; Full time

Benefits offered by this job

Remote work

Flexible scheduling

Competitive hourly pay

Paid expert network opportunities

Job summary

prolificacademicltd seeks experienced AI/ML engineers to join a paid expert network for evaluating and auditing language model behavior. This remote, task-based role emphasizes applying deep technical judgment to critique models rather than building production systems.

You will review explanations of architectures, audit training loops, provide feedback for alignment and safety, and help benchmark model reasoning with quantitative metrics and RLHF workflows. Flexible scheduling is offered.

Qualifications

• Advanced degree in CS/AI with ML focus.
• Hands-on experience deploying or fine-tuning ML models.
• Strong knowledge of Transformers, CNNs, RNNs and optimization.
• Experience with prompt engineering and RLHF or retrieval-augmented generation workflows.
• Ability to audit complex model logic and evaluate proofs behind ML algorithms.

Responsibilities

• Review explanations of model architectures for accuracy.
• Audit ML code, training loops, preprocessing, and evaluation pipelines.
• Provide high-quality human feedback to align models with human intent, safety, and helpfulness through RLHF workflows.
• Analyze reasoning in chain-of-thought prompts and pinpoint where logic breaks down.
• Run benchmarks between model outputs using technical taxonomies and quantitative metrics.

Skills

ML model development

Transformers

RLHF

Prompt engineering

Model evaluation

Education

MS/PhD in CS/AI/related field

BS in CS or related field

Tools

PyTorch

TensorFlow/Keras

Pandas

NumPy

Weights & Biases

LangChain

Pinecone/Milvus/Weaviate

Job description

prolificacademicltd seeks experienced AI/ML engineers to join a paid expert network for evaluating and auditing language model behavior. This remote, task-based role emphasizes applying deep technical judgment to critique models rather than building production systems.

You will review explanations of architectures, audit training loops, provide feedback for alignment and safety, and help benchmark model reasoning with quantitative metrics and RLHF workflows. Flexible scheduling is offered.