AI Benchmark Quality Auditor & Feedback Specialist

eDataBae; Colombia; On-site; COP 202,007,000 - 303,010,000; Full time

Job summary

eDataBae is seeking experienced AI Benchmark Quality Reviewers to support an advanced AI evaluation project. You will review evaluation tasks, analyze model outputs, validate grading logic, and provide evidence-based feedback to improve benchmark quality.

The role requires 5+ years of relevant experience, strong analytical skills, and the ability to clearly document findings with reproducible evidence in English. This position is based in Colombia and focuses on improving AI solution reliability.

Qualifications

• 5+ years of relevant professional experience is required.
• Ability to read and understand Python, SQL, and shell scripts.
• Strong analytical and problem-solving skills with attention to detail.

Responsibilities

• Review task instructions, source materials, reference solutions, and evaluation criteria for accuracy and completeness.
• Analyze AI agent execution traces, tool calls, and generated outputs.
• Identify issues in grading logic, expected answers, and evaluation criteria.

Skills

Data analysis

Technical quality assurance

AI evaluation

Tools

Python

SQL

Shell scripting

Job description

eDataBae is seeking experienced AI Benchmark Quality Reviewers to support an advanced AI evaluation project. You will review evaluation tasks, analyze model outputs, validate grading logic, and provide evidence-based feedback to improve benchmark quality.

The role requires 5+ years of relevant experience, strong analytical skills, and the ability to clearly document findings with reproducible evidence in English. This position is based in Colombia and focuses on improving AI solution reliability.