Remote Intelligence Analyst for AI Evaluation

Executive Staff Recruiters / ESR Healthcare; United States; Remote; USD 55,000 - 96,000; Full time

Job summary

micro1 is seeking an Intelligence Analyst contractor to support a remote, customer-facing project focused on evaluating and advancing AI capabilities. You will map analytic cases, author test scenarios, and assess AI-generated outputs to ensure high-quality benchmarks and learning signals for models.

Your expertise in all-source analysis and analytic rigor will guide the calibration of evaluation standards and the refinement of procedures across the team.

Qualifications

• Experience in all-source analysis, civilian-IC background preferred.
• Proficient in source evaluation and confidence-level assessment.
• Experience teaching or mentoring analytic reviews.
• Excellent written and verbal communication.

Responsibilities

• Develop and map analytic cases to defined taxonomies across multiple evaluation areas.
• Author complex test cases to evaluate model performance on source handling, analytic rigor, and uncertainty expression.
• Assess and grade AI-generated responses based on reasoning, source use, limitations, and practical utility.
• Identify and flag instances where analytic exploration is over-refused by the model, with documentation and feedback.
• Document edge cases and analytic disagreements to support evaluation methodology improvements.
• Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
• Contribute insights to calibrate analytic standards and improve AI system benchmarks.

Skills

All-source analytics

Source evaluation

Confidence-level assessment

Analytic reviews

Written and verbal communication

Job description

micro1 is seeking an Intelligence Analyst contractor to support a remote, customer-facing project focused on evaluating and advancing AI capabilities. You will map analytic cases, author test scenarios, and assess AI-generated outputs to ensure high-quality benchmarks and learning signals for models.

Your expertise in all-source analysis and analytic rigor will guide the calibration of evaluation standards and the refinement of procedures across the team.