Remote ML/AI Peer Review Evaluator

Join; Zürich; Hybrid; CHF 83.000 - 138.000; Teilzeit

Benefits dieser Stelle

Fully remote

Part-time contract

Zusammenfassung

Lightly AG, a Zurich-based AI company, seeks researchers with strong ML/AI backgrounds to support an AI evaluation project focusing on scientific peer review. You will evaluate agentic AI reviews against expert human peer reviews, using a structured rubric and evidence-based rationales.

This is a remote, project-based contractor opportunity with flexible hours. Part-time arrangement preferred for ongoing evaluation across multiple papers.

Qualifikationen

• Advanced analytical thinking and ability to interpret complex ML/AI content.
• Strong written communication skills for clear evaluation rationales.
• Excellent attention to detail when identifying factual inaccuracies or hallucinations.

Aufgaben

• Read and scan ML/AI research papers to extract core contributions and experiments.
• Review original human peer reviews to establish baselines for each paper.
• Evaluate AI-generated peer reviews against the expert baseline using a structured rubric.
• Assess technical accuracy, depth, constructive value, and novelty in each AI review.
• Identify hallucinations, unsupported claims, and missed technical issues.
• Compare two AI reviews to determine which provides stronger analysis.
• Search and verify relevant literature using Google Scholar, arXiv, or Semantic Scholar.
• Provide concise, evidence-based rationales applying the project rubric consistently.

Kenntnisse

Analytical thinking

Written communication

Attention to detail

Ausbildung

Master’s degree or PhD in ML/AI/CS

Jobbeschreibung

Lightly AG, a Zurich-based AI company, seeks researchers with strong ML/AI backgrounds to support an AI evaluation project focusing on scientific peer review. You will evaluate agentic AI reviews against expert human peer reviews, using a structured rubric and evidence-based rationales.

This is a remote, project-based contractor opportunity with flexible hours. Part-time arrangement preferred for ongoing evaluation across multiple papers.