Remote Intelligence Analyst – AI Evaluation & Analytics

OpenTrain AI, Inc.; United States; Remote; USD 41,000 - 110,000; Part time

Benefits offered by this job

Remote work

Part-time contract

Job summary

OpenTrain AI, Inc. is seeking a remote, part-time contractor to evaluate AI-generated responses, test source handling, and analytic reasoning, applying intelligence tradecraft and standards.

You will map analytic cases to defined taxonomies, write test cases, and assess responses against criteria, documenting model limits and uncertainties. English fluency is required, and 20+ hours per week are typical.

Qualifications

• English fluency required.
• Entry level experience.
• Knowledge: source evaluation, confidence-level assessment, established analytic standards, structured analytic techniques, and intelligence tradecraft.

Responsibilities

• Map analytic cases to defined taxonomies across several intelligence evaluation areas.
• Write complex test cases that examine source use, analytic rigor, uncertainty, and practical usefulness.
• Assess and grade AI-generated responses against clear analytic criteria.
• Identify and document over-refusal, edge cases, analytic disagreements, and model limitations.
• Help refine source-evaluation procedures and confidence-level assignments.
• Share expert feedback that improves analytic standards and AI evaluation benchmarks.

Skills

English fluency

Job description

OpenTrain AI, Inc. is seeking a remote, part-time contractor to evaluate AI-generated responses, test source handling, and analytic reasoning, applying intelligence tradecraft and standards.

You will map analytic cases to defined taxonomies, write test cases, and assess responses against criteria, documenting model limits and uncertainties. English fluency is required, and 20+ hours per week are typical.