Remote ML Researcher: AI Peer-Review Evaluator

Lightly; Schweiz; Hybrid; EUR 83.000 - 138.000; Teilzeit

Benefits dieser Stelle

Fully remote

Part-time contractor role

Flexible hours

Work on cutting-edge AI evaluation

Zusammenfassung

Lightly AG, a Zurich-based AI company, is seeking researchers to support an AI evaluation project focused on scientific peer review. You will assess AI-generated reviews against expert human peer reviews of ML/AI research papers.

This is a remote, project-based contractor opportunity with flexible hours. You will evaluate reviews, identify gaps, and provide concise rationales using a structured rubric across multiple papers.

Qualifikationen

• Advanced degree in ML/AI or related field; evidence of scholarship.
• Ability to critically read ML/AI research papers including methodology and results.
• Experience performing literature searches and verifying prior work dates.

Aufgaben

• Read ML/AI papers to identify core contributions, methods, experiments and claims.
• Review original human peer reviews to establish a baseline.
• Evaluate AI-generated reviews against the baseline using a structured rubric.
• Assess accuracy, depth, constructiveness, and novelty of AI reviews.
• Identify hallucinations, unsupported claims, and missed issues.
• Compare two AI reviews side-by-side and judge usefulness.
• Search and verify relevant literature using Google Scholar, arXiv, etc.
• Provide concise, evidence-based rationales per rubric.

Kenntnisse

Master's degree in ML/AI

PhD candidate or holder

Experience reviewing ML/AI papers

Literature search skills

Analytical writing

Familiar with NeurIPS ICML ICLR ACL CV

Attention to detail

Ausbildung

Master's degree in ML/AI/CS

PhD in ML/AI

Tools

Google Scholar

arXiv

Semantic Scholar

Jobbeschreibung

Lightly AG, a Zurich-based AI company, is seeking researchers to support an AI evaluation project focused on scientific peer review. You will assess AI-generated reviews against expert human peer reviews of ML/AI research papers.

This is a remote, project-based contractor opportunity with flexible hours. You will evaluate reviews, identify gaps, and provide concise rationales using a structured rubric across multiple papers.