Remote Operations Architect for AI Benchmarking

EVB; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

EVB is seeking an experienced operations professional to design multi-step, gradable benchmark tasks drawn from real operations work. The role focuses on creating tasks with objective criteria that enable consistent evaluation by reviewers and require judgment over simple lookup.

You will assemble reference materials, spreadsheets, and process documents to support each task. The ideal candidate will have 5+ years in operations, reporting, and process design, strong Excel skills, and excellent

Qualifications

• 5+ years of hands-on responsibility in business operations, operations reporting, process design, or workflow management.
• Experience creating or applying structured standards: SOPs, QA checklists, process documentation, service level definitions, or training assessments.
• Strong Excel capability and comfort reasoning across multiple documents at once.
• Strong English, written and spoken — B2 minimum, C1 preferred.
• AI or LLM evaluation experience is a plus, but not required.

Responsibilities

• Design multi-step tasks drawn from real operations work — reporting, process design, workflow management — with realistic reference files and clear instructions.
• Define objective grading criteria so two qualified reviewers would reach the same verdict on the same output.
• Build tasks that require judgment rather than lookup: ones where the right answer depends on reading the materials correctly.
• Create or source the supporting materials each task needs: spreadsheets, system exports, process documentation, operational reports.
• Flag tasks that turn out to be ambiguous or ungradable instead of submitting them anyway.
• Deliver approved tasks on schedule and at the required quality level.

Skills

Excel

English language

Operations reporting

Process design

AI/LLM evaluation

Tools

SOP development

Job description

EVB is seeking an experienced operations professional to design multi-step, gradable benchmark tasks drawn from real operations work. The role focuses on creating tasks with objective criteria that enable consistent evaluation by reviewers and require judgment over simple lookup.

You will assemble reference materials, spreadsheets, and process documents to support each task. The ideal candidate will have 5+ years in operations, reporting, and process design, strong Excel skills, and excellent