AI Research Scientist – Scientific Evaluation (ML/AI)

Jobgether; United States; Remote; USD 69,000 - 90,000; Part time

Benefits offered by this job

Fully remote opportunity

Freelance / independent contractor

Part-time, flexible hours

Competitive hourly rate $50–$65

Job summary

Jobgether is seeking an AI Research Scientist focused on scientific evaluation to assess AI-generated research for rigor, originality, and methodological soundness. You will critique experimental design, baselines, and reproducibility, delivering concise, evidence-based feedback.

This fully remote, freelance, part-time role requires strong publication credentials and the ability to verify citations, results, and code where available.

Qualifications

• PhD or equivalent demonstrated research expertise in ML/AI.
• Extensive hands-on ML/AI research experience and publishing original work.
• Ability to assess novelty, significance, and technical validity of research.

Responsibilities

• Critically evaluate AI-generated ML/AI research for scientific contributions, rigor, and originality.
• Assess experimental design, baselines, metrics, and statistical validity; review code when available.
• Identify methodological flaws, unsupported claims, and discrepancies between claims and evidence.

Skills

PhD in ML/AI

NeurIPS/ICML/ICLR publications

Independent research

Experiment design

Reproducibility

Education

PhD in ML/AI/CS/Stats

Job description

Accountabilities:

As an AI Research Scientist specializing in scientific evaluation, you will critically examine AI-generated research outputs and determine whether they demonstrate the scientific rigor and originality expected at top-tier machine learning and AI conferences. Your responsibilities will include:

• Evaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and the validity of their conclusions.
• Assess the novelty and significance of proposed approaches by comparing them with existing research and state-of-the-art methods.
• Examine experimental rigor, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies.
• Identify methodological flaws, unsupported claims, inconsistencies, misleading results, and other issues that could compromise scientific validity.
• Assess reproducibility and technical soundness by reviewing experimental evidence, implementation details, and accompanying code when available.
• Verify relevant academic literature, identify overlooked prior work, check citation accuracy, and challenge unsupported novelty claims.
• Determine whether research outputs meet the scientific standards associated with leading ML/AI conferences, including NeurIPS, ICML, and ICLR.
• Produce structured, evidence-based evaluations outlining research strengths, weaknesses, critical technical concerns, and overall scientific merit.
• Apply detailed assessment rubrics consistently while providing clear, concise, and well-reasoned justifications for your evaluations.
• Exercise independent expert judgment when determining whether AI-generated research represents a genuine scientific contribution and is supported by sufficient evidence.

Requirements:

• A PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research expertise.
• At least three peer-reviewed research publications at NeurIPS, ICML, and/or ICLR. This is a mandatory requirement.
• Ideally, first-author publications or other significant research contributions demonstrating substantial scientific involvement.
• Extensive hands‑on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original research.
• Strong knowledge of scientific methodology and experimental rigor, including baseline selection, statistical validity, benchmark design, ablation studies, and reproducibility.
• Ability to independently assess the novelty, significance, technical soundness, and scientific validity of advanced ML/AI research.
• Familiarity with state-of-the-art research and the ability to evaluate new approaches against established literature and benchmarks.
• Experience reviewing research papers for leading ML/AI conferences or journals, particularly NeurIPS, ICML, or ICLR; this is strongly preferred.
• Ability to verify technical arguments, experimental evidence, scientific claims, citations, and code implementations where relevant.
• Excellent analytical skills and scientific writing abilities, with the capacity to communicate nuanced technical judgments through clear, concise, evidence-based feedback.
• Ability to apply detailed evaluation guidelines and scoring rubrics consistently while maintaining independent scientific judgment.
• Exceptional attention to detail, particularly when identifying subtle methodological weaknesses, unsupported conclusions, and discrepancies between claims and evidence.
• Willingness to provide a CV, a brief overview of your research background and areas of expertise, and links to at least three qualifying peer‑reviewed publications.
• Relevant academic peer‑review experience, including service as a reviewer or area chair at major research conferences, should be highlighted where applicable.

Benefits:

• Fully remote opportunity open to candidates worldwide.
• Competitive compensation of $50–$65 per hour .
• Freelance or independent contractor arrangement.
• Part‑time, project‑based work with flexible hours.
• Opportunity to evaluate cutting‑edge autonomous AI systems capable of conducting end‑to‑end scientific research.
• Chance to apply your research expertise to the evaluation and advancement of frontier AI agents.
• Exposure to emerging developments in AI‑driven scientific discovery.
• Opportunity to contribute specialist scientific judgment to research evaluation processes focused on technical validity, originality, and experimental rigor.
• Flexible working arrangements that support independent research professionals.