Senior AI Auditor & RLHF Evaluation Expert

prolificacademicltd; EE. UU.; A distancia; USD 91.000 - 129.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Remote work

Flexible hours

Short onboarding assessment

Descripción de la vacante

prolificacademicltd is seeking a senior AI/ML engineer to join a paid participant opportunity based in Memphis but remote-friendly. You will apply production ML experience to evaluate frontier LLMs, emphasizing technical judgment, code auditing, and analytical critique rather than model development.

Responsibilities include auditing architectures, loss functions and backpropagation explanations, ML code, and providing feedback for RLHF; tracing reasoning through prompts; and benchmarking outputs

Formación

• BS, MS or PhD in CS/AI/Robotics or quantitative ML field.
• Production ML experience building, deploying or fine-tuning ML systems.
• Deep knowledge of Transformer/CNN/RNN architectures and optimizations.
• Hands-on with prompt engineering, RLHF or retrieval-augmented workflows.

Responsabilidades

• Scrutinize AI explanations of architectures, loss functions and backpropagation for accuracy.
• Audit ML code, including training loops, data preprocessing and notebooks, flagging issues.
• Provide high-quality feedback for RLHF on alignment, safety and usefulness.
• Trace AI reasoning through chain-of-thought prompts to locate failures.
• Benchmark model outputs using defined metrics and taxonomies.

Conocimientos

Production ML

Transformers

Prompt engineering

RLHF

Code auditing

Analytical reasoning

Neural networks

Educación

BS/MS/PhD in CS/AI/Robotics or quantitative field with ML concentration

Herramientas

PyTorch

TensorFlow/Keras

Hugging Face Transformers

Pinecone/Milvus/Weaviate

SageMaker/Vertex AI

Descripción del empleo

prolificacademicltd is seeking a senior AI/ML engineer to join a paid participant opportunity based in Memphis but remote-friendly. You will apply production ML experience to evaluate frontier LLMs, emphasizing technical judgment, code auditing, and analytical critique rather than model development.

Responsibilities include auditing architectures, loss functions and backpropagation explanations, ML code, and providing feedback for RLHF; tracing reasoning through prompts; and benchmarking outputs