AI QA Trainer - LLM Evaluation

Meridial

United States

Remote

USD 8,300 - 90,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Aitrainergigs is seeking an AI QA trainer to shape the evaluation of large-scale language models. You will design test plans, run regression suites, and document failure modes to improve prompts, guardrails, and evaluation metrics. This role emphasizes model reliability, safety, and high-signal bug reporting.

You will collaborate on adversarial red-teaming, automation with Python/SQL, and dashboards to track quality deltas over time in a remote, contract setting.

Qualifications

  • Bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or related field ideal.

Responsibilities

  • Converse with the model on real-world scenarios and evaluation prompts.
  • Verify factual accuracy and logical soundness of responses.

Skills

Model evaluation
LLM safety
Prompt robustness
Data quality assurance
Multilingual testing
Domain-specific testing
Grounding verification
Compliance checks
Adversarial red-teaming
Automation (Python/SQL)
Dashboarding

Education

Bachelor’s/Master’s/PhD in CS or related field

Tools

PyTest
OpenAI Evals
RAG evaluators
Weights & Biases (W&B)
Python
SQL

Job description

Are you an AI QA expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into enterprise-grade platforms. With rigorous evaluation data, tomorrow’s AI can democratize world-class education, keep pace with cutting-edge research, and streamline workflows for teams everywhere. That quality begins with you—we need your expertise to harden model reasoning and reliability.

We’re looking for AI QA trainers who live and breathe model evaluation, LLM safety, prompt robustness, data quality assurance, multilingual and domain-specific testing, grounding verification, and compliance/readiness checks. You’ll challenge advanced language models on tasks like hallucination detection, factual consistency, prompt-injection and jailbreak resistance, bias/fairness audits, chain-of-reasoning reliability, tool-use correctness, retrieval-augmentation fidelity, and end-to-end workflow validation—documenting every failure mode so we can raise the bar.

On a typical day, you will converse with the model on real-world scenarios and evaluation prompts, verify factual accuracy and logical soundness, design and run test plans and regression suites, build clear rubrics and pass/fail criteria, capture reproducible error traces with root-cause hypotheses, and suggest improvements to prompt engineering, guardrails, and evaluation metrics (e.g., precision/recall, faithfulness, toxicity, and latency SLOs). You’ll also partner on adversarial red-teaming, automation (Python/SQL), and dashboarding to track quality deltas over time.

A bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field is ideal; shipped QA for ML/AI systems, safety/red-team experience, test automation frameworks (e.g., PyTest), and hands-on work with LLM eval tooling (e.g., OpenAI Evals, RAG evaluators, W&B) signal fit. Skills that stand out include: evaluation rubric design, adversarial testing/red-teaming, regression testing at scale, bias/fairness auditing, grounding verification, prompt and system-prompt engineering, test automation (Python/SQL), and high-signal bug reporting. Clear, metacognitive communication—“showing your work”—is essential.

We offer a pay range of $6-to- $65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high-speed internet; company-sponsored benefits such as health insurance and PTO do not apply.

Employment type: Contract

Workplace type: Remote

Seniority level: Mid-Senior Level

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI QA Trainer
AI QA Trainer

Meridial • United States

Remote
USD 8,300 - 90,000
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Meridial • United States

Remote
USD 8,265 - 89,544
Secure computer and high-speed internet required
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Agency • United States

Remote
USD 8,300 - 90,000
AI QA Trainer – LLM Evaluation
AI QA Trainer – LLM Evaluation

Invisible Technologies • United States

Remote
USD 8,300 - 90,000
Remote AI QA Trainer & Evaluation Specialist
Remote AI QA Trainer & Evaluation Specialist

Meridial • United States

Remote
USD 8,265 - 89,544
Secure computer and high-speed internet required
AI QA Trainer - Freelance Project
AI QA Trainer - Freelance Project

agency • United States

Remote
USD 8,300 - 90,000
Remote contract
Flexible hourly rate
AI QA Trainer
AI QA Trainer

Agency • United States

Remote
USD 120,000 - 160,000
Remote AI QA Trainer: LLM Evaluation & Reliability
Remote AI QA Trainer: LLM Evaluation & Reliability

Agency • United States

Remote
USD 8,300 - 90,000
AI QA Trainer: LLM Evaluation & Safety Expert
AI QA Trainer: LLM Evaluation & Safety Expert

Agency • United States

Remote
USD 120,000 - 160,000
AI Test Engineer (LLM and AI QA)
AI Test Engineer (LLM and AI QA)

Synoptek • United States

Remote
USD 90,000 - 120,000