AI Research Scientist – Scientific Evaluation (ML/AI)

Jobgether; Netherlands; Remote; EUR 62,000 - 80,000; Part time

Benefits offered by this job

Fully remote worldwide

Hourly compensation

Independent contractor

Part-time project-based work

Job summary

Jobgether is seeking an AI Research Scientist – Scientific Evaluation (ML/AI) based in the Netherlands for a remote freelance role. You will evaluate AI research papers against conference standards, assess novelty, rigor, reproducibility, and provide evidence-based feedback.

This part-time, contract position offers flexible hours and competitive hourly pay, with compensation of $50-65 per hour. You will work independently using structured rubrics to judge scientific merit and contribute to

Qualifications

• PhD or equivalent research expertise in ML/AI/CS or related field.
• At least three peer‑reviewed publications at major ML/AI venues.
• Strong experimental design, rigor, and analytical writing skills.
• Ability to assess novelty, significance, and scientific validity.
• Experience reviewing NeurIPS/ICML/ICLR papers is preferred.
• Willingness to share CV and publication links and provide evidence-based feedback.

Responsibilities

• Evaluate AI papers for contributions, correctness, and reproducibility.
• Assess novelty and significance against state‑of‑the‑art methods.
• Examine experimental rigor, baselines, metrics, and ablations.
• Identify methodological weaknesses and unsupported claims.
• Verify literature, citations, and code implementations when available.
• Produce structured, evidence-based evaluations with clear justifications.
• Apply evaluation rubrics while maintaining independent scientific judgment.

Skills

Peer review experience

ML/AI expertise

Scientific writing

Experimental design

Statistical methods

Independent judgment

Education

PhD in ML/AI/CS or equivalent

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an AI Research Scientist – Scientific Evaluation (ML/AI) based in Netherlands.

Join an advanced AI research evaluation project focused on assessing scientific work produced by autonomous AI research agents. In this remote, freelance role, you will apply your expertise in machine learning and artificial intelligence to evaluate research papers against the rigorous standards of leading academic conferences. You will assess scientific novelty, technical correctness, experimental design, reproducibility, and the validity of research conclusions. Your expert judgment will help distinguish meaningful scientific advances from flawed methodologies or unsupported claims. Working independently with structured evaluation guidelines, you will contribute to the development and assessment of frontier AI systems capable of conducting end-to-end scientific research. This opportunity offers flexible hours, competitive hourly compensation, and the chance to apply your research expertise to emerging AI capabilities.

Accountabilities:

• Evaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and the validity of their conclusions.


• Assess the novelty and significance of proposed approaches by comparing them with existing research and state-of-the-art methods.


• Examine experimental rigor, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies.


• Identify methodological flaws, unsupported claims, inconsistencies, misleading results, and other issues that could compromise scientific validity.


• Assess reproducibility and technical soundness by reviewing experimental evidence, implementation details, and accompanying code when available.


• Verify relevant academic literature, identify overlooked prior work, check citation accuracy, and challenge unsupported novelty claims.


• Determine whether research outputs meet the scientific standards associated with leading ML/AI conferences, including NeurIPS, ICML, and ICLR.


• Produce structured, evidence-based evaluations outlining research strengths, weaknesses, critical technical concerns, and overall scientific merit.


• Apply detailed assessment rubrics consistently while providing clear, concise, and well-reasoned justifications for your evaluations.


• Exercise independent expert judgment when determining whether AI-generated research represents a genuine scientific contribution and is supported by sufficient evidence.

Requirements:

• A PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research expertise.


• At least three peer-reviewed research publications at NeurIPS, ICML, and/or ICLR. This is a mandatory requirement.


• Ideally, first-author publications or other significant research contributions demonstrating substantial scientific involvement.


• Extensive hands‑on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original research.


• Strong knowledge of scientific methodology and experimental rigor, including baseline selection, statistical validity, benchmark design, ablation studies, and reproducibility.


• Ability to independently assess the novelty, significance, technical soundness, and scientific validity of advanced ML/AI research.


• Familiarity with state-of-the-art research and the ability to evaluate new approaches against established literature and benchmarks.


• Experience reviewing research papers for leading ML/AI conferences or journals, particularly NeurIPS, ICML, or ICLR; this is strongly preferred.


• Ability to verify technical arguments, experimental evidence, scientific claims, citations, and code implementations where relevant.


• Excellent analytical skills and scientific writing abilities, with the capacity to communicate nuanced technical judgments through clear, concise, evidence-based feedback.


• Ability to apply detailed evaluation guidelines and scoring rubrics consistently while maintaining independent scientific judgment.


• Exceptional attention to detail, particularly when identifying subtle methodological weaknesses, unsupported conclusions, and discrepancies between claims and evidence.


• Willingness to provide a CV, a brief overview of your research background and areas of expertise, and links to at least three qualifying peer‑reviewed publications.


• Relevant academic peer‑review experience, including service as a reviewer or area chair at major research conferences, should be highlighted where applicable.

Benefits:

• Fully remote opportunity open to candidates worldwide.


• Competitive compensation of $50-65 per hour.


• Freelance or independent contractor arrangement.


• Part‑time, project‑based work with flexible hours.


• Opportunity to evaluate cutting‑edge autonomous AI systems capable of conducting end‑to‑end scientific research.


• Chance to apply your research expertise to the evaluation and advancement of frontier AI agents.


• Exposure to emerging developments in AI‑driven scientific discovery.


• Opportunity to contribute specialist scientific judgment to research evaluation processes focused on technical validity, originality, and experimental rigor.


• Flexible working arrangements that support independent research professionals.

How Jobgether works:

We use an AI‑powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top‑fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre‑contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.