Research Evaluation Specialist (PhD or Masters / Researcher / Professor)

AuraOne, Inc.; United States; Remote; USD 55,000 - 110,000; Full time

Benefits offered by this job

PhD/academic experience preferred

Experience reviewing AI tools

Multilingual fluency (non-English)

Job summary

AuraOne, Inc. is seeking a Research Evaluation Specialist (PhD or Masters / Researcher / Professor) to remotely review AI outputs across evaluation methods and workflows. You will reproduce derivations, flag errors, and document methods to guide the modeling team.

Responsibilities include grading, adjudicating disputes using textbooks or papers, and maintaining inter-rater calibration scores. This role is a contractor with hourly compensation determined after interview.

Qualifications

• Graduate-level training or equivalent applied experience in research evaluation or related field.
• Hands-on experience publishing, teaching, or advising on the topic at a professional level.
• Ability to apply multi-page rubrics consistently across long batches.

Responsibilities

• Review AI outputs against research evaluation methods and prior work.
• Reproduce or sanity-check key derivations and claims.
• Flag dimensional or methodological errors with structured severity tags.
• Capture corrected reasoning or worked examples for model training.
• Adjudicate disputed answers against textbooks or papers.
• Maintain reviewer quality scores in calibration cycles.

Skills

Scientific reasoning

Method validation

Citation review

Quantitative analysis

AI model evaluation

Research evaluation

Education

Graduate degree in related field

Publications/teaching experience

Job description

Research Evaluation Specialist (PhD or Masters / Researcher / Professor) is a remote review track for evaluating AI outputs across research evaluation research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method so the modeling team can train on it.

Why this role matters

Research Evaluation research review models live or die on whether their derivations actually hold up under scrutiny. AuraOne uses scientific specialists to grade outputs the way a peer reviewer would — checking assumptions, reproducing key steps, and capturing the right method alongside the wrong one.

Responsibilities

• Review AI outputs against current research evaluation research review methods, conventions, and prior work for Research Evaluation Specialist (PhD or Masters / Researcher / Professor) assignments.
• Reproduce or sanity-check key derivations, calculations, or experimental claims.
• Flag dimensional, methodological, and citation errors with structured severity tags.
• Capture the corrected reasoning or worked example so the modeling team can train on it.
• Adjudicate disputed answers against textbooks, papers, or community standards.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Qualifications

• Graduate-level training or equivalent applied experience in research evaluation research review or a closely related field for Research Evaluation Specialist (PhD or Masters / Researcher / Professor) work.
• Hands-on experience publishing, teaching, or advising on the topic at a professional level.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that cites methods, papers, or worked examples.
• Reliable async availability for at least 10 hours per week.

Example tasks

• Reproduce a research evaluation research review derivation from a model output and flag any algebraic or dimensional errors.
• Grade a model's literature summary against the cited papers and rate the citation quality.
• Adjudicate a disputed answer between two reviewers using textbook methods.
• Audit a 25-row batch for rubric consistency and report drift to the program lead.

Nice to have

• PhD, postdoc, or industry research experience in the topic area.
• Prior work reviewing AI-assisted research tooling and its failure modes.
• Multilingual fluency for non-English papers and corpora.

Skills

• Scientific reasoning
• Method validation
• Citation review
• Quantitative analysis
• Research Evaluation research review
• AI model evaluation

Work model

Remote - US-eligible. Remote Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

Hourly rate confirmed after the interview process.

Graduate-level training or equivalent applied experience in research evaluation research review or a closely related field for Research Evaluation Specialist (PhD or Masters / Researcher / Professor) work. Hands-on experience publishing, teaching, or advising on the topic at a professional level. Comfort applying multi-page rubrics consistently across long batches. Clear written reasoning that cites methods, papers, or worked examples. Reliable async availability for at least 10 hours per week.