Remote Senior Rater for AI Model Evaluation & Calibration

pofft Technology Inc.

United States

Remote

USD 69,000 - 165,000

Full time

7 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

pofft Technology Inc. is seeking a senior rater to join a fully remote expert network, evaluating complex AI model outputs and calibrating other graders. You will rank and rate expert-level model answers using rubrics and write clear rationales.

Work spans mathematics, coding, law, medicine, and finance, with tasks including fact-checking, policy adherence, and developing grading guidelines. Compensation ranges from $50 to $120 per hour.

Qualifications

  • At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain.
  • Native or near-native written English proficiency.
  • Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
  • Experience leading or reviewing other raters, such as as a QA lead, rubric owner, or reviewer of reviewers.
  • Hands-on AI data annotation experience, search-quality rating, or experience writing and maintaining grader guidelines is a plus.

Responsibilities

  • Rank and rate expert-level model answers using detailed rubrics and write clear rationales.
  • Evaluate steps in model reasoning across fields such as mathematics, coding, law, medicine, and finance.
  • Fact-check complex claims against primary sources and verify accuracy.
  • Assess answers against content policies, style guides, and edge cases.
  • Review other raters' work, calibrate standards, and develop or refine grading guidelines.
  • Assess search relevance and provide feedback in live sessions.

Skills

Quality assurance
Finance
Mathematics
Content writing

Education

Advanced degree

Job description

pofft Technology Inc. is seeking a senior rater to join a fully remote expert network, evaluating complex AI model outputs and calibrating other graders. You will rank and rate expert-level model answers using rubrics and write clear rationales.

Work spans mathematics, coding, law, medicine, and finance, with tasks including fact-checking, policy adherence, and developing grading guidelines. Compensation ranges from $50 to $120 per hour.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Expert Talent Pool Feedback and Alignment
Expert Talent Pool Feedback and Alignment

VettedBench • United States

Remote
USD 69,000 - 165,000
Remote work
Competitive pay
Flexible hours
Expert Talent Pool, Feedback and Alignment (Senior Expert)
Expert Talent Pool, Feedback and Alignment (Senior Expert)

Clera • United States

Remote
USD 69,000 - 165,000
Fully remote
Expert Talent Pool Feedback and Alignment
Expert Talent Pool Feedback and Alignment

Clera • United States

Remote
USD 69,000 - 165,000
Fully remote
Flexible project durations
Senior AI Rater & Reviewer — Remote Calibration Expert
Senior AI Rater & Reviewer — Remote Calibration Expert

Clera • United States

Remote
USD 69,000 - 165,000
Fully remote
Senior Rater & AI Feedback Lead (Remote)
Senior Rater & AI Feedback Lead (Remote)

Clera • United States

Remote
USD 69,000 - 165,000
Fully remote
Flexible project durations
Expert Talent Pool Feedback and Alignment
Expert Talent Pool Feedback and Alignment

pofft Technology Inc. • United States

Remote
USD 69,000 - 165,000
Senior Expert: Model Answer Evaluator & Calibration Lead
Senior Expert: Model Answer Evaluator & Calibration Lead

Clera • United States

Remote
USD 69,000 - 165,000
Competitive pay
Fully remote
Expert Talent Pool, Feedback and Alignment (Senior Expert)
Expert Talent Pool, Feedback and Alignment (Senior Expert)

Clera • United States

Remote
USD 69,000 - 165,000
Competitive pay
Fully remote
CS Grad for AI Model Evaluation - Remote & Flexible
CS Grad for AI Model Evaluation - Remote & Flexible

prolificacademicltd • United States

Remote
USD 28,000 - 83,000
Remote work
Flexible hours
Diverse assignments
Remote AI Evaluator (CS Grad) - Model Reasoning, Up to $60/h
Remote AI Evaluator (CS Grad) - Model Reasoning, Up to $60/h

prolificacademicltd • United States

Remote
USD 68,000 - 97,000
Remote work
Flexible participation