Remote AI Quality Engineer: RLHF & Model Audits

prolificacademicltd; United States; Remote; USD 91,000 - 129,000; Part time

Benefits offered by this job

Remote work

Flexible hours

Competitive hourly rates up to $80

Job summary

Prolific Academic Ltd invites experienced AI and ML engineers to contribute to training and evaluating next-generation large language models. Work is performed remotely as part of an expert network, with flexible paid engagements for independent professionals based in or near Milwaukee.

Responsibilities include reviewing explanations of model architectures and loss functions, auditing training loops and data preprocessing, and providing high-quality feedback for RLHF workflows to improve

Qualifications

• BS/MS/PhD in CS/AI with ML focus.
• Hands-on ML model development and deployment experience.
• Experience with Transformers, CNNs, and RNNs.
• Ability to audit model logic and identify data contamination.

Responsibilities

• Review explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit ML code and notebooks, including training loops and data preprocessing.
• Provide high-quality feedback for RLHF workflows to improve alignment with human intent.
• Evaluate chain-of-thought reasoning to identify breakdowns or flawed outputs.
• Benchmark model outputs against defined technical taxonomies and metrics.

Skills

Transformers

RNNs

CNNs

RLHF

Prompt engineering

Retrieval-augmented generation

Python

ML production deployment

Education

BS in Computer Science or related field

MS or PhD in CS/AI/related field

Tools

PyTorch

TensorFlow/Keras

Hugging Face Transformers

AWS SageMaker

LangChain

Pinecone

Milvus

Job description

Prolific Academic Ltd invites experienced AI and ML engineers to contribute to training and evaluating next-generation large language models. Work is performed remotely as part of an expert network, with flexible paid engagements for independent professionals based in or near Milwaukee.

Responsibilities include reviewing explanations of model architectures and loss functions, auditing training loops and data preprocessing, and providing high-quality feedback for RLHF workflows to improve