Remote AI QA Trainer — LLM Evaluation & Safety

Meridial

United States

Remote

USD 8,300 - 90,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

AI Trainer HQ is seeking an AI QA trainer to evaluate language models and build robust testing pipelines. You will design test plans, write regression suites, and document failures with root-cause hypotheses while improving prompts, guardrails, and evaluation metrics.

The role emphasizes multilingual and domain-specific testing, grounding verification, and compliance checks, with remote work and Singapore-based placement. Prior ML QA, safety/red-team and automation experience is valued.

Qualifications

  • Shipped QA for ML/AI systems; safety/red-team experience.
  • Test automation frameworks (e.g., PyTest) and LLm eval tooling experience.
  • Clear, metacognitive communication—showing your work—essential.

Responsibilities

  • Evaluate prompts and model outputs for accuracy and safety.
  • Design and run test plans and regression suites.
  • Build rubrics and pass/fail criteria; document failures with root-cause hypotheses.
  • Collaborate on adversarial red-teaming, automation, and dashboards to track quality deltas.

Skills

Model evaluation
LLM safety
Prompt robustness
Data quality
Multilingual testing
Grounding verification
Compliance checks
Adversarial testing
Red-teaming
Python/SQL
Dashboarding
Test automation
Root-cause analysis
Prompt engineering

Education

Bachelor's degree
Master's degree
PhD

Tools

PyTest
OpenAI Evals
RAG evaluators
W&B

Job description

AI Trainer HQ is seeking an AI QA trainer to evaluate language models and build robust testing pipelines. You will design test plans, write regression suites, and document failures with root-cause hypotheses while improving prompts, guardrails, and evaluation metrics.

The role emphasizes multilingual and domain-specific testing, grounding verification, and compliance checks, with remote work and Singapore-based placement. Prior ML QA, safety/red-team and automation experience is valued.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI QA Trainer: LLM Evaluation & Reliability
Remote AI QA Trainer: LLM Evaluation & Reliability

Agency • United States

Remote
USD 8,300 - 90,000
Remote AI QA Trainer: Evaluation & Safety Lead
Remote AI QA Trainer: Evaluation & Safety Lead

Meridial • United States

Remote
USD 8,300 - 90,000
AI QA Trainer: LLM Evaluation & Safety Expert
AI QA Trainer: LLM Evaluation & Safety Expert

Agency • United States

Remote
USD 120,000 - 160,000
AI QA Trainer - Freelance Project
AI QA Trainer - Freelance Project

Meridial • United States

Remote
USD 8,300 - 90,000
AI QA Trainer - LLM Evaluation
AI QA Trainer - LLM Evaluation

Meridial • United States

Remote
USD 8,300 - 90,000
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Meridial • United States

Remote
USD 8,265 - 89,544
Secure computer and high-speed internet required
Remote AI QA Trainer for Model Safety & Quality
Remote AI QA Trainer for Model Safety & Quality

agency • United States

Remote
USD 8,300 - 90,000
Remote contract
Flexible hourly rate
Remote AI QA Trainer & Evaluation Expert
Remote AI QA Trainer & Evaluation Expert

Invisible Technologies Inc. • United States

Remote
USD 8,300 - 90,000
AI QA Trainer
AI QA Trainer

Meridial • United States

Remote
USD 8,300 - 90,000
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Agency • United States

Remote
USD 8,300 - 90,000