Remote Senior AI/ML Engineer — LLM Evaluation & Audits

prolificacademicltd; United States; Remote; USD 82,656,000 - 110,208,000; Part time

Benefits offered by this job

Remote, flexible contributor model

Paid hourly tasks up to $80/hr

Job summary

prolificacademic ltd is seeking a Senior AI/ML Engineer to help train and evaluate large language models on a remote, project-based basis. You will provide high-quality technical judgement to research teams building next-generation AI systems, with focus on reliability, efficiency, and safety.

Responsibilities include auditing model code and RLHF workflows, reviewing explanations, and benchmarking outputs against technical taxonomies, while collaborating with researchers across locations.

Qualifications

• BS, MS, or PhD in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field with a machine learning focus.
• Hands-on experience building, deploying, or fine-tuning ML models in production environments.
• Professional-level understanding of neural network architectures (Transformers, CNNs, RNNs) and optimization techniques.
• Practical experience with prompt engineering, RLHF, or retrieval-augmented generation (RAG) workflows.
• Ability to audit complex model logic, detect training data contamination, and evaluate mathematical proofs.
• Sharp attention to detail for spotting hallucinations and biased or flawed AI-generated content.

Responsibilities

• Review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
• Audit machine learning code, including training loops, data preprocessing scripts, and evaluation notebooks, for correctness and efficiency.
• Supply structured human feedback used to refine reinforcement learning from human feedback (RLHF) pipelines.
• Critically assess chain-of-thought reasoning, flagging hallucinations, biases, and logical breakdowns in model outputs.
• Run comparative benchmarks across model outputs against defined technical taxonomies and performance metrics.

Skills

ML model production

Prompt engineering

RLHF workflows

Code audit

Model evaluation

Education

BS/MS/PhD in CS/AI/Robotics

PhD preferred

Tools

PyTorch

TensorFlow/Keras

Hugging Face Transformers

AWS SageMaker

Vertex AI

Weights & Biases

LangChain

Pinecone

Milvus

Weaviate

Job description

prolificacademic ltd is seeking a Senior AI/ML Engineer to help train and evaluate large language models on a remote, project-based basis. You will provide high-quality technical judgement to research teams building next-generation AI systems, with focus on reliability, efficiency, and safety.

Responsibilities include auditing model code and RLHF workflows, reviewing explanations, and benchmarking outputs against technical taxonomies, while collaborating with researchers across locations.