Intelligence Analyst - Remote

YO AI Labs

New York (NY)

Remote

USD 69,000 - 96,000

Part time

Just now
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

YO AI Labs is seeking an experienced Intelligence Analyst to contribute to evaluating and improving next-generation AI systems. You will apply intelligence analysis skills to assess AI outputs, evaluate reasoning, and help improve accuracy, reliability, and consistency of AI models.

Remote contractor role requiring strong analytical judgment, clear communication, and the ability to document edge cases and provide actionable feedback to the project leads for continuous improvement.

Qualifications

  • All-source intelligence analysis experience or civilian IC expertise.
  • Strong source evaluation, confidence-level assessment, and established intelligence analysis standards.
  • Experience teaching, mentoring, or leading analytical reviews.
  • Excellent written and verbal communication with clarity, precision, and evidence-based reasoning.
  • Ability to identify and explain nuanced model limitations, including cases of unnecessary refusal.
  • Familiarity with structured analytical techniques and intelligence tradecraft standards.
  • Previous experience in AI evaluation, model assessment, or related quality assurance initiatives is a plus.

Responsibilities

  • Develop and map analytical cases to taxonomies across evaluation areas.
  • Create complex test cases to evaluate model performance in source evaluation and reasoning.
  • Assess and grade AI-generated responses based on reasoning quality and source use.
  • Document edge cases and ambiguous scenarios to support continuous improvement.
  • Collaborate with project leads to refine procedures for source evaluation and confidence-level assessment.
  • Provide insights to calibrate analytical standards and improve AI evaluation benchmarks.

Skills

Intelligence analysis
Source evaluation
Analytical rigor
Communication
Model limitations
Structured analytic techniques
AI evaluation

Job description

Job Summary

We are seeking experienced Intelligence Analysts to contribute to a project focused on evaluating and improving next-generation AI systems. In this role, you will apply your intelligence analysis expertise to assess AI-generated outputs, evaluate analytical reasoning, and help strengthen the accuracy, reliability, and consistency of AI models. Prior experience in AI is not required; relevant analytical expertise is the primary requirement.

Job Title: Intelligence Analyst

Role Type: Contractor

Location: Remote

Key Responsibilities
  • Develop and map analytical cases to established taxonomies across multiple evaluation areas, ensuring comprehensive coverage of relevant intelligence scenarios.
  • Create complex test cases to evaluate model performance in source evaluation, analytical rigor, and the expression of uncertainty.
  • Assess and grade AI-generated responses based on reasoning quality, source use, recognition of limitations, and practical utility.
  • Identify instances where legitimate analytical exploration is unnecessarily refused by the model, providing clear documentation and actionable feedback.
  • Document edge cases, analytical disagreements, and ambiguous scenarios to support continuous improvement of evaluation methodologies.
  • Collaborate with project leads to refine procedures for source evaluation, confidence-level assessment, and analytical review.
  • Provide insights to help calibrate analytical standards and improve the overall quality and consistency of AI evaluation benchmarks.
Preferred Qualifications
  • Demonstrated all-source intelligence analysis experience or deep expertise in a specialized analytical discipline, ideally within a civilian Intelligence Community (IC) environment.
  • Strong proficiency in source evaluation, confidence-level assessment, and established intelligence analysis standards.
  • Experience teaching, mentoring, or leading analytical reviews, with a proven ability to promote analytical rigor.
  • Excellent written and verbal communication skills, with a strong emphasis on clarity, precision, and evidence-based reasoning.
  • Ability to identify and explain nuanced model limitations, including cases of unnecessary refusal.
  • Familiarity with structured analytical techniques and intelligence tradecraft standards.
  • Previous experience in AI evaluation, model assessment, or related quality assurance initiatives is an advantage.
Ideal Candidate Profile

The ideal candidate combines strong intelligence analysis expertise with sound judgment, critical thinking, and attention to detail. They can evaluate complex analytical outputs, assess source credibility, communicate uncertainty effectively, and identify gaps in reasoning. Their feedback will help improve the reliability, consistency, and analytical quality of AI-generated intelligence assessments.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Intelligence Analyst
Intelligence Analyst

confidential • United States

Remote
USD 55,000 - 96,000
Intelligence Analyst remote
Intelligence Analyst remote

Executive Staff Recruiters / ESR Healthcare • United States

Remote
USD 55,000 - 96,000
Intelligence Analyst
Intelligence Analyst

Micro1 • United States

Remote
USD 70,000 - 110,000
Refer-a-friend bonus
AI Evaluation Intelligence Analyst - Remote
AI Evaluation Intelligence Analyst - Remote

YO AI Labs • New York (NY)

Remote
USD 69,000 - 96,000
Senior Intelligence Analyst – National Security
Senior Intelligence Analyst – National Security

confidential • United States

Remote
USD 83,000 - 152,000
Intelligence Agency Officer - Remote
Intelligence Agency Officer - Remote

YO AI Labs • New York (NY)

Remote
USD 62,000 - 117,000
Senior Intelligence Analyst – National Security
Senior Intelligence Analyst – National Security

Remotebridge • United States

Remote
USD 96,000 - 152,000
Remote Intelligence Analyst - AI Benchmark Evaluator
Remote Intelligence Analyst - AI Benchmark Evaluator

Executive Staff Recruiters / ESR Healthcare • United States

Remote
USD 55,000 - 96,000
Data Analyst (AI Evaluation)
Data Analyst (AI Evaluation)

micro1 Inc. • United States

Remote
USD 55,000 - 96,000
Senior Intelligence Analyst - National Security (Remote)
Senior Intelligence Analyst - National Security (Remote)

Micro1 • United States

Remote
USD 69,000 - 138,000