Senior ML Auditor for RLHF & AI Alignment (Remote)

prolificacademicltd; United States; Remote; USD 91,000 - 129,000; Full time

Benefits offered by this job

Remote work

Flexible task-based work

Competitive hourly pay

Job summary

prolificacademicltd is offering a remote, paid expert network role for Senior machine learning engineers to audit, review, and provide structured feedback on LLM outputs and ML code. The position emphasizes methodological rigor and alignment with human expectations.

Responsibilities include auditing explanations, ensuring technical fidelity, and delivering high-quality judgments that improve RLHF safety and usefulness across model evaluations.

Qualifications

• BS/MS/PhD in CS/AI with ML focus.
• Experience deploying or fine-tuning ML models.
• Strong knowledge of transformers and optimization techniques.
• Hands-on prompt engineering and RLHF/RAG pipelines.
• Ability to critique model behavior, detect contamination, and assess mathematical underpinnings.

Responsibilities

• Critically review AI explanations of architectures, loss functions, and backpropagation for technical fidelity.
• Audit ML codebases and notebooks, including training loops and evaluation scripts, for correctness and performance.
• Deliver high-quality human judgments that drive RLHF alignment with respect to safety, helpfulness, and intent.
• Trace and critique chain-of-thought reasoning to surface breakdowns in model logic.
• Conduct systematic benchmark comparisons across model outputs using technical rubrics and metrics.

Skills

Audit ML code and models

RLHF alignment

Prompt engineering

Editorial accuracy

Education

Bachelor’s/Master’s/PhD in CS/AI/ML

Tools

PyTorch

TensorFlow/Keras

Hugging Face Transformers

LangChain

AWS SageMaker / Vertex AI

Job description

prolificacademicltd is offering a remote, paid expert network role for Senior machine learning engineers to audit, review, and provide structured feedback on LLM outputs and ML code. The position emphasizes methodological rigor and alignment with human expectations.

Responsibilities include auditing explanations, ensuring technical fidelity, and delivering high-quality judgments that improve RLHF safety and usefulness across model evaluations.