AI QA Trainer - Freelance Project

Meridial

United States

Remote

USD 8.300 - 90.000

Vollzeit

14 Tage+

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

AI Trainer HQ is seeking an AI QA trainer to evaluate language models and build robust testing pipelines. You will design test plans, write regression suites, and document failures with root-cause hypotheses while improving prompts, guardrails, and evaluation metrics.

The role emphasizes multilingual and domain-specific testing, grounding verification, and compliance checks, with remote work and Singapore-based placement. Prior ML QA, safety/red-team and automation experience is valued.

Qualifikationen

  • Shipped QA for ML/AI systems; safety/red-team experience.
  • Test automation frameworks (e.g., PyTest) and LLm eval tooling experience.
  • Clear, metacognitive communication—showing your work—essential.

Aufgaben

  • Evaluate prompts and model outputs for accuracy and safety.
  • Design and run test plans and regression suites.
  • Build rubrics and pass/fail criteria; document failures with root-cause hypotheses.
  • Collaborate on adversarial red-teaming, automation, and dashboards to track quality deltas.

Kenntnisse

Model evaluation
LLM safety
Prompt robustness
Data quality
Multilingual testing
Grounding verification
Compliance checks
Adversarial testing
Red-teaming
Python/SQL
Dashboarding
Test automation
Root-cause analysis
Prompt engineering

Ausbildung

Bachelor's degree
Master's degree
PhD

Tools

PyTest
OpenAI Evals
RAG evaluators
W&B

Jobbeschreibung

Are you an AI QA expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into enterprise-grade platforms. With rigorous evaluation data, tomorrow’s AI can democratize world-class education, keep pace with cutting-edge research, and streamline workflows for teams everywhere. That quality begins with you—we need your expertise to harden model reasoning and reliability.

We’re looking for AI QA trainers who live and breathe model evaluation, LLM safety, prompt robustness, data quality assurance, multilingual and domain‑specific testing, grounding verification, and compliance/readiness checks. You’ll challenge advanced language models on tasks like hallucination detection, factual consistency, prompt‑injection and jailbreak resistance, bias/fairness audits, chain‑of‑reasoning reliability, tool‑use correctness, retrieval‑augmentation fidelity, and end‑to‑end workflow validation—documenting every failure mode so we can raise the bar.

On a typical day, you will converse with the model on real‑world scenarios and evaluation prompts, verify factual accuracy and logical soundness, design and run test plans and regression suites, build clear rubrics and pass/fail criteria, capture reproducible error traces with root‑cause hypotheses, and suggest improvements to prompt engineering, guardrails, and evaluation metrics (e.g., precision/recall, faithfulness, toxicity, and latency SLOs). You’ll also partner on adversarial red‑teaming, automation (Python/SQL), and dashboarding to track quality deltas over time.

A bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field is ideal; shipped QA for ML/AI systems, safety/red‑team experience, test automation frameworks (e.g., PyTest), and hands‑on work with LLM eval tooling (e.g., OpenAI Evals, RAG evaluators, W&B) signal fit. Skills that stand out include: evaluation rubric design, adversarial testing/red‑teaming, regression testing at scale, bias/fairness auditing, grounding verification, prompt and system‑prompt engineering, test automation (Python/SQL), and high‑signal bug reporting. Clear, metacognitive communication—“showing your work”—is essential.

We offer a pay range of $6-to- $65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company-sponsored benefits such as health insurance and PTO do not apply.

Employment type: Contract

Workplace type: Remote

Seniority level: Mid‑Senior Level

Languages
  • English

This job is 100% remote.

Open only to people in Singapore.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

AI QA Trainer
AI QA Trainer

Meridial • USA

Remote
USD 8.300 - 90.000
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Meridial • USA

Remote
USD 8.265 - 89.544
Secure computer and high-speed internet required
AI QA Trainer - LLM Evaluation
AI QA Trainer - LLM Evaluation

Meridial • USA

Remote
USD 8.300 - 90.000
AI QA Trainer – LLM Evaluation
AI QA Trainer – LLM Evaluation

Invisible Technologies • USA

Remote
USD 8.300 - 90.000
AI QA Trainer - LLM Evaluation - Freelance Project
AI QA Trainer - LLM Evaluation - Freelance Project

Agency • USA

Remote
USD 8.300 - 90.000
AI QA Trainer - Freelance Project
AI QA Trainer - Freelance Project

agency • USA

Remote
USD 8.300 - 90.000
Remote contract
Flexible hourly rate
Remote AI QA Trainer & Evaluation Specialist
Remote AI QA Trainer & Evaluation Specialist

Meridial • USA

Remote
USD 8.265 - 89.544
Secure computer and high-speed internet required
Remote AI QA Trainer — LLM Evaluation & Safety
Remote AI QA Trainer — LLM Evaluation & Safety

Meridial • USA

Remote
USD 8.300 - 90.000
AI QA Trainer
AI QA Trainer

Agency • USA

Remote
USD 120.000 - 160.000
English Quality Assurance Lead (QAL)
English Quality Assurance Lead (QAL)

SuperAnnotate AI, Inc. • USA

Remote
USD 29.000 - 48.000
Remote contractor role