Remote AI Security Operations QA Analyst

AI Trainer Jobs; United States; Remote; USD 28,000 - 55,000; Part time

Job summary

AI Trainer Jobs is seeking an independent contractor for remote evaluation of SOC analyst AI training prompts and outputs. You will review frontier model results, apply the rubric, and generate structured feedback for retraining.

A strong writer with attention to detail is essential, as is the ability to work asynchronously across multiple batches. The role is remote from the United States, with hourly compensation to be confirmed after interview.

Qualifications

• Attention to detail in evaluating model outputs and labeling.
• Clear written reasoning naming issues and rubric clauses.
• Reliable asynchronous availability for the role (10+ hours/week).
• Experience with inter-rater calibration and rubric-based evaluation.
• Domain expertise in cybersecurity or SOC analysis is beneficial.

Responsibilities

• Evaluate SOC analyst AI training outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them for rubric updates.

Skills

Attention to detail

Clear written reasoning

Reliable async availability

Inter-rater calibration

Domain expertise in cybersecurity

Job description

AI Trainer Jobs is seeking an independent contractor for remote evaluation of SOC analyst AI training prompts and outputs. You will review frontier model results, apply the rubric, and generate structured feedback for retraining.

A strong writer with attention to detail is essential, as is the ability to work asynchronously across multiple batches. The role is remote from the United States, with hourly compensation to be confirmed after interview.