ntelligence Analyst

Executive Staff Recruiters / ESR Healthcare; United States; Remote; USD 55,000 - 96,000; Full time

Job summary

micro1 is seeking an Intelligence Analyst contractor to support a remote, customer-facing project focused on evaluating and advancing AI capabilities. You will map analytic cases, author test scenarios, and assess AI-generated outputs to ensure high-quality benchmarks and learning signals for models.

Your expertise in all-source analysis and analytic rigor will guide the calibration of evaluation standards and the refinement of procedures across the team.

Qualifications

• Experience in all-source analysis, civilian-IC background preferred.
• Proficient in source evaluation and confidence-level assessment.
• Experience teaching or mentoring analytic reviews.
• Excellent written and verbal communication.

Responsibilities

• Develop and map analytic cases to defined taxonomies across multiple evaluation areas.
• Author complex test cases to evaluate model performance on source handling, analytic rigor, and uncertainty expression.
• Assess and grade AI-generated responses based on reasoning, source use, limitations, and practical utility.
• Identify and flag instances where analytic exploration is over-refused by the model, with documentation and feedback.
• Document edge cases and analytic disagreements to support evaluation methodology improvements.
• Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
• Contribute insights to calibrate analytic standards and improve AI system benchmarks.

Skills

All-source analytics

Source evaluation

Confidence-level assessment

Analytic reviews

Written and verbal communication

Job description

ntelligence Analyst

Role Type: Contractor

Location: Remote

micro1 is engaging Intelligence Analysts to contribute to a customer-facing project focused on evaluating and advancing artificial intelligence capabilities. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

1. Develop and map analytic cases to defined taxonomies across multiple evaluation areas, ensuring comprehensive coverage of relevant intelligence scenarios.
2. Author complex test cases designed to evaluate model performance on source handling, analytic rigor, and nuanced uncertainty expression.
3. Assess and grade AI-generated responses based on criteria such as reasoning, source use, recognition of limitations, and practical utility.
4. Identify and flag instances where legitimate analytic exploration is over-refused by the model, providing clear documentation and feedback.
5. Document edge cases and analytic disagreements, supporting continuous improvement of evaluation methodologies.
6. Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
7. Contribute insights to help calibrate analytic standards and enhance the overall quality of AI system benchmarks.

Preferred Qualifications

1. Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline, ideally with a civilian-IC background.
2. Strong proficiency in source evaluation, confidence-level assessment, and established analytic standards.
3. Experience in teaching, mentoring, or leading analytic reviews, with a record of fostering analytic rigor.
4. Exceptional written and verbal communication skills, with a focus on clarity and precision.
5. Ability to identify and articulate nuanced model limitations and over-refusal scenarios.
6. Familiarity with structured analytic techniques and intelligence tradecraft standards.
7. Previous participation in AI evaluation or model assessment initiatives is a plus.