Remote Intelligence Analyst: AI Evaluation & Benchmarking

Zekai; United States; À distance; USD 55 000 - 96 000; Temps partiel

Résumé du poste

Zekai is seeking an Intelligence Analyst for a 100% remote contract role. You will contribute to evaluating and advancing AI capabilities, training models through high-quality real-world input, and shaping how systems learn and reason.

Experience in AI evaluation is helpful, but domain knowledge matters most. As a contractor, you will develop analytic cases, grade AI responses, and document edge cases while collaborating with project leads to refine evaluation procedures and standards on a

Qualifications

• Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline is preferred.
• Strong ability to evaluate sources and assign confidence levels to analyses.
• Experience teaching or leading analytic reviews is a plus.
• Excellent written and spoken communication with precision and clarity.
• Ability to articulate model limitations and over-refusal scenarios clearly.

Responsabilités

• Develop and map analytic cases to defined taxonomies across evaluation areas.
• Author complex test cases to evaluate AI model performance on source handling and uncertainty.
• Assess and grade AI-generated responses for reasoning and practicality.
• Identify when analytic exploration is over-refused by the model and document feedback.
• Document edge cases and analytic disagreements to improve methodologies.
• Collaborate with project leads to refine evaluation procedures and confidence scoring.
• Contribute insights to calibrate analytic standards and overall benchmark quality.

Connaissances

All-source analysis

Analytic rigor

Source evaluation

Mentoring/lead analytic reviews

Structured analytic techniques

Formation

Civilian-IC background

Description du poste

Zekai is seeking an Intelligence Analyst for a 100% remote contract role. You will contribute to evaluating and advancing AI capabilities, training models through high-quality real-world input, and shaping how systems learn and reason.

Experience in AI evaluation is helpful, but domain knowledge matters most. As a contractor, you will develop analytic cases, grade AI responses, and document edge cases while collaborating with project leads to refine evaluation procedures and standards on a