Production AI Evaluation Specialist

OpenTrain AI; Northern (KY); Hybrid; USD 62,000 - 117,000; Part time

Job summary

OpenTrain AI is seeking a production expert to design realistic evaluation tasks for AI training. You will translate production judgment into structured scenarios covering budgets, schedules, contracts, call sheets and crew assignments. This remote, part-time contractor role requires 20+ hours per week.

Compensation ranges from $45 to $85 USD per hour. Five+ years of credited production experience and strong written English are required to document tasks, rubrics, and benchmarks for AI

Qualifications

• At least five years of credited experience in film, television, digital, or live event production.
• Direct experience managing and reconciling production budgets, call sheets, and contracts.
• Ability to coordinate vendors and crew while balancing time, cost, and resource constraints.
• Experience creating realistic production artifacts, defining correct decisions, or evaluating multi-criteria rubrics.
• Strong written and verbal communication skills for production documentation and technical materials.

Responsibilities

• Design production-management evaluation scenarios, including budget reconciliation and scheduling adjustments.
• Source, construct, and curate authentic budgets, contracts, call sheets, delivery schedules, and crew assignments.
• Define correct decisions and comprehensive solutions that account for multiple production constraints.
• Develop rubrics with 35 or more criteria for evaluating deliverables, judgment, and constraint management.
• Document production processes and workflows in clear technical English.

Skills

Production experience

Budget reconciliation

Vendor coordination

English communication

Job description

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We connect experienced professionals with specialized opportunities where their practical judgment helps shape how modern AI systems are evaluated and improved.

Creating an OpenTrain account is free. You can build a profile that showcases your experience, discover relevant projects, and apply in minutes while developing a lasting portfolio of AI training work.

About AI Training Work

AI training is the human side of building artificial intelligence. Professionals create examples, review model outputs, and design structured evaluations that help AI systems perform more reliably in real-world situations.

In this role, your production expertise will be translated into realistic scenarios and assessment criteria. Your work will help evaluate whether AI-generated decisions properly account for budgets, schedules, contracts, vendors, crews, and other production constraints.

The Role

OpenTrain is seeking a film, television, digital, or live event production expert to create realistic evaluation tasks for AI training. You will convert production-management judgment into structured scenarios involving budget reconciliation, scheduling adjustments, contracts, call sheets, delivery planning, vendor coordination, and crew assignments.

This is a part-time contractor engagement requiring 20 or more hours per week. The work is remote and asynchronous, with compensation ranging from $45 to $85 USD per hour.

• Category: Production AI evaluation and text generation
• Experience: Five or more years of credited production experience
• Schedule: 20 or more hours per week
• Engagement: Part-time contractor
• Language: English
• Compensation: $45 to $85 USD per hour

What You’ll Do

You will design tasks that simulate the practical decisions production professionals make under competing constraints. Each task should reflect realistic industry workflows and provide enough context to assess nuanced reasoning and deliverable quality.

You will work with project collaborators to refine task designs while maintaining credible production standards. Documentation must be clear and precise enough for AI benchmarking and technical review.

• Design production-management evaluation scenarios, including budget reconciliation and scheduling adjustments.
• Source, construct, and curate authentic budgets, contracts, call sheets, delivery schedules, and crew assignments.
• Define correct decisions and comprehensive solutions that account for multiple production constraints.
• Develop rubrics with 35 or more criteria for evaluating deliverables, judgment, and constraint management.
• Document production processes and workflows in clear technical English.
• Refine evaluation tasks with collaborators while preserving realistic industry standards.

Requirements

You should have substantial hands‑on production experience and be comfortable making decisions that balance time, cost, resources, and operational realities. The role requires both professional production judgment and the ability to express that judgment through structured assessments.

Strong written and verbal English communication is essential because you will create technical materials and document workflows for evaluation and benchmarking.

• At least five years of credited experience in film, television, digital, or live event production.
• Direct experience managing and reconciling production budgets, call sheets, and contracts.
• Ability to coordinate vendors and crew while balancing time, cost, and resource constraints.
• Experience creating realistic production artifacts, defining correct decisions, or evaluating multi‑criteria rubrics.
• Strong written and verbal communication skills for production documentation and technical materials.
• Strong analytical judgment and the ability to work independently in a remote, asynchronous engagement.

Helpful Background

Experience creating realistic industry benchmarks, evaluating complex production decisions, or translating professional workflows into structured assessment materials can help you succeed. Familiarity with rubric-based evaluation is especially relevant to the task design involved in this project.

• Production benchmark or assessment development
• Evaluation of complex production decisions
• Workflow documentation and technical writing
• Multi-criteria rubric authoring or review

Why Work With OpenTrain

AI training and data-labeling work is a fast-growing way to participate in technology without leaving your professional specialty behind. Contributors with deep industry knowledge help determine whether AI systems understand and respond to real operational challenges.

OpenTrain gives you a place to build a credible AI training profile, find projects aligned with your expertise, and grow specialized experience over time. This role lets you apply production knowledge to cutting‑edge AI evaluation while working remotely and managing a flexible part‑time schedule.

• Work remotely with an asynchronous engagement.
• Apply professional production judgment to emerging AI systems.
• Build an AI training portfolio around specialized expertise.
• Choose a part‑time workload of 20 or more hours per week.