AI Evaluation Architect for Benchmarking

Crossing Hurdles; Indonesia; On-site; IDR 133,920,000 - 200,880,000; Full time

Job summary

Crossing Hurdles is seeking an evaluator to design and implement self-contained AI evaluation tasks, including prompts, supporting files, and detailed rubrics. You will define unambiguous success criteria and meticulously observe AI behavior, producing precise English reports.

The role requires fluency in English writing, strong attention to detail, and experience with rubric-based evaluation. Collaboration with the customer’s team is essential for ongoing improvement.

Qualifications

• Native or fluent in English writing with concise, specific observations.
• Experience designing rubric-based evaluation or scoring frameworks.
• High attention to detail and ability to spot subtle patterns.
• Excellent written and verbal communication for documenting feedback.

Responsibilities

• Design and implement self-contained evaluation tasks with prompts, supporting files, and detailed grading rubrics.
• Define clear written criteria for successful and unsuccessful task completion across diverse scenarios.
• Observe and document AI agent behaviors, producing precise summaries and reports.
• Iterate evaluation tasks and rubrics based on feedback to ensure robust benchmarking.
• Collaborate with the customer’s team to share insights and drive continuous improvement.

Skills

Written precision

Structured thinking

Attention to detail

English writing

Tools

Rubric design

Evaluation frameworks

Job description

Crossing Hurdles is seeking an evaluator to design and implement self-contained AI evaluation tasks, including prompts, supporting files, and detailed rubrics. You will define unambiguous success criteria and meticulously observe AI behavior, producing precise English reports.

The role requires fluency in English writing, strong attention to detail, and experience with rubric-based evaluation. Collaboration with the customer’s team is essential for ongoing improvement.