Baseball Fan – Live MLB Game AI Evaluator (US)

AI Trainer Jobs; United States; Remote; USD 28,000 - 41,000; Part time

Job summary

AI Trainer Jobs in the United States seeks individuals with strong baseball knowledge to evaluate AI assistants during MLB postseason games. You will ask questions live, compare two AI products, capture conversations, and rate responses for usefulness and accuracy.

Applicants should be US-based with at least 1 year of professional experience, superb English, and attention to detail. Familiarity with AI tools and web search is a plus, with game-day windows required.

Qualifications

• Based in the United States
• At least 1 year of professional work experience
• Solid baseball knowledge: rules, teams, players, postseason basics
• Avid MLB fan or sports enthusiast is a plus
• Available to work live during game windows on scheduled game days
• Strong written English, sharp attention to detail, and ability to spot factual errors
• Prior AI rating or evaluation experience is a plus but not required

Responsibilities

• Follow live MLB postseason games and ask AI assistants questions as a fan would
• Run the same question on two AI products side by side, as a single turn or 2–10 turns
• Capture share links and full conversation screenshots
• Rate responses for helpfulness, accuracy, and presentation, and write a clear rationale

Skills

Baseball knowledge

Strong written English

Attention to detail

Spot factual errors

AI tools usage

Web search

Tools

AI chat assistants

Job description

Role overview

Mercor is looking for US-based people with strong baseball knowledge to evaluate AI assistants live, while real MLB games are being played. On scheduled game days, you'll ask AI assistants the questions you'd naturally ask while following a game, then rate how helpful, accurate, and well-presented their answers are. Your judgments directly shape how AI products serve sports fans.

What you'll do

• Follow live MLB postseason games and ask AI assistants real fan questions: live scores and win probability, player and team stats, rules and challenges, schedules and broadcasts, highlights, and expert-style analysis
• Run the same question on two AI products side by side, either as a single question or as a multi-turn conversation of 2 to 10 turns
• Capture share links and full screenshots of every conversation
• Rate each response for helpfulness, factual accuracy, and presentation, choose which product did better, and write a clear rationale for your choice

What we're looking for

• Based in the United States
• At least 1 year of professional work experience
• Solid baseball knowledge: rules, teams, players, and how the postseason works
• Being an avid MLB fan or a general sports enthusiast is a plus
• Available to work live during game windows on scheduled game days
• Comfortable using AI chat assistants and web search
• Strong written English, sharp attention to detail, and the ability to spot factual errors
• Prior AI rating or evaluation experience is a plus, but not required

Details

• Event-based: work happens on MLB postseason game days in October, from the Division Series through the World Series
• Each task takes about 2 to 2.5 hours and is completed while the game is live
• The first game day is as early as Saturday, October 3

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.