AI Evaluation Specialist – Video Games Domain

eDataBae; United States; On-site; USD 70,000 - 110,000; Full time

Job summary

eDataBae is seeking experienced professionals in the video game industry to support AI evaluation initiatives. The role involves creating gaming-focused evaluation scenarios, reviewing AI-generated responses, and providing structured feedback to improve AI systems.

Candidates should have deep knowledge of game development, esports, platforms, and publishing, with strong written English, analytical skills, and the ability to work independently on research tasks.

Qualifications

• Strong knowledge of the video game industry, including game development, esports, platforms, publishers, and gaming communities.
• Excellent written English and analytical skills.
• Strong attention to detail and ability to evaluate information objectively.
• Ability to research, analyze, and provide structured feedback.
• Experience working independently on research-based activities.

Responsibilities

• Create domain-specific prompts related to video games, game development, esports, platforms, publishers, and gaming communities.
• Evaluate AI-generated responses for factual accuracy, completeness, reasoning quality, and relevance.
• Identify incorrect information, inconsistencies, outdated knowledge, and edge cases.
• Compare AI responses and provide evidence-based feedback.
• Support the development of evaluation datasets and testing scenarios.
• Maintain high-quality documentation and evaluation standards.

Skills

Video game knowledge

Written English

Analytical skills

Attention to detail

Research ability

Independent work

Education

Advanced studies in video games

Tools

Generative AI familiarity

Job description

About the Opportunity

We are looking for experienced professionals with strong expertise in the video game industry to support AI evaluation initiatives.

The role involves creating gaming-related evaluation scenarios, reviewing AI-generated responses, and providing structured feedback to improve the quality, accuracy, and reasoning capabilities of AI systems.

Key Responsibilities

• Create domain-specific prompts related to video games, game development, esports, platforms, publishers, and gaming communities
• Evaluate AI-generated responses for factual accuracy, completeness, reasoning quality, and relevance
• Identify incorrect information, inconsistencies, outdated knowledge, and edge cases
• Compare AI responses and provide evidence-based feedback
• Support the development of evaluation datasets and testing scenarios
• Maintain high-quality documentation and evaluation standards

Required Skills & Experience

• Strong knowledge of the video game industry, including game development, esports, platforms, publishers, and gaming communities
• Excellent written English and analytical skills
• Strong attention to detail and ability to evaluate information objectively
• Ability to research, analyze, and provide structured feedback
• Experience working independently on research-based activities

Preferred Qualifications

• Advanced academic background in video games, game studies, interactive media, or related fields
• 3 years of professional, research, teaching, or industry experience
• Experience with Generative AI, Large Language Models (LLMs), prompt engineering, or AI evaluation is a plus
• Experience creating written analysis, reviews, or research content