Remote Intelligence Analyst for AI Evaluation and Analytics

micro1; United States; Remote; USD 70,000 - 110,000; Part time

Benefits offered by this job

Refer-a-friend bonus

Job summary

micro1 is a data lab that helps AI labs and Enterprises train foundational models and agents, offering frontier evaluations and reinforcement learning environments to improve LLM capabilities. We also provide contextual evaluations to monitor and improve AI agents in enterprise settings. Role: Intelligence Analyst (Contractor) | Location: Remote.

No prior AI experience required; domain knowledge matters. You will develop analytic tests, evaluate model responses, and help refine evaluation

Qualifications

• All-source analytic experience or civilian-IC background preferred.
• Strong source evaluation, confidence-level assessment, and analytic standards.
• Experience teaching, mentoring, or leading analytic reviews.
• Exceptional written and verbal communication skills.
• Ability to articulate nuanced model limitations and over-refusal scenarios.
• Familiarity with structured analytic techniques and intelligence tradecraft.
• Experience in AI evaluation or model assessment is a plus.
• Email

Responsibilities

• Develop and map analytic cases to defined taxonomies across multiple evaluation areas.
• Author complex test cases to evaluate model performance on source handling, analytic rigor, and nuanced uncertainty expression.
• Assess and grade AI-generated responses based on reasoning, source use, recognition of limitations, and practical utility.
• Identify and flag instances where legitimate analytic exploration is over-refused by the model, providing clear documentation and feedback.
• Document edge cases and analytic disagreements, supporting continuous improvement of evaluation methodologies.
• Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
• Contribute insights to help calibrate analytic standards and enhance the overall quality of AI system benchmarks.

Skills

All-source analytics

Source evaluation

Confidence-level assessment

Analytic standards

Mentoring analytic reviews

Communication skills

Model limitations

AI evaluation experience

Email

Job description

micro1 is a data lab that helps AI labs and Enterprises train foundational models and agents, offering frontier evaluations and reinforcement learning environments to improve LLM capabilities. We also provide contextual evaluations to monitor and improve AI agents in enterprise settings. Role: Intelligence Analyst (Contractor) | Location: Remote.

No prior AI experience required; domain knowledge matters. You will develop analytic tests, evaluate model responses, and help refine evaluation