AI Evaluation Architect: Administrative Workflows

OpenTrain AI, Inc.; Northern (KY); Hybrid; USD 60,000 - 90,000; Full time

Job summary

OpenTrain AI, Inc. is seeking an Administrative Workflow AI Evaluation Expert to design expert-authored evaluation tasks for a frontier AI benchmark.

You will translate administrative challenges into structured scenarios that test how AI systems coordinate tools, follow processes, and complete real-world operations. Candidates with five+ years in professional administration, strong written and verbal communication, and attention to detail will thrive.

Qualifications

• At least five years of experience supporting teams in a professional administrative role.
• Ownership of ongoing, recurring workflows and documented procedures.
• Strong written and verbal communication and professional correspondence.
• Comfort with CSVs, PDFs, spreadsheets, and technical documents.

Responsibilities

• Design process-driven evaluation tasks for admin benchmarking.
• Assemble realistic artifacts like emails, calendar entries, receipts, notes.
• Define solutions and document step-by-step decisions for experts.
• Build rubrics with 35+ criteria on process adherence and judgment.

Skills

Administrative experience

Communication skills

Attention to detail

Professional correspondence

Tools

CSV

PDF

Spreadsheets

Technical documents

Job description

OpenTrain AI, Inc. is seeking an Administrative Workflow AI Evaluation Expert to design expert-authored evaluation tasks for a frontier AI benchmark.

You will translate administrative challenges into structured scenarios that test how AI systems coordinate tools, follow processes, and complete real-world operations. Candidates with five+ years in professional administration, strong written and verbal communication, and attention to detail will thrive.