Remote AI QA Trainer & Evaluation Expert

Invisible Technologies Inc.; United States; Remote; USD 8,300 - 90,000; Part time

Job summary

Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability across multilingual and domain-specific testing.

You will design test plans, build rubrics, and document failure modes to raise standards for safety, bias checks, and prompt robustness. As a contractor, you will work remotely with flexible hours, collaborating with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose improvements to prompts, guardrails, and evaluation

Qualifications

• For ideal fit, advanced degrees or certifications in ML/AI testing, safety red-teaming, and test automation.
• Hands-on experience with LLM eval tooling (OpenAI Evals, RAG evaluators, W&B) signals fit.
• Shipped QA for ML/AI systems and familiarity with evaluation rubrics.

Responsibilities

• Converse with the model on real-world scenarios and evaluation prompts.
• Verify factual accuracy and logical soundness of outputs.
• Design and run test plans and regression suites.
• Build clear rubrics and pass/fail criteria for evaluations.
• Capture reproducible error traces with root-cause hypotheses.
• Suggest improvements to prompt engineering, guardrails, and metrics.
• Partner on adversarial red-teaming, automation (Python/SQL), and dashboards.

Skills

Model evaluation

LLM safety

Prompt robustness

Data quality assurance

Multilingual testing

Domain-specific testing

Grounding verification

Compliance readiness checks

Hallucination detection

Factual consistency

Prompt-injection resistance

Jailbreak resistance

Bias/fairness audits

Chain-of-reasoning reliability

Tool-use correctness

Retrieval-augmentation fidelity

End-to-end workflow validation

Adversarial red-teaming

Python/SQL automation

Education

Bachelor's degree (CS/Data Science/Computational Linguistics/Statistics)

Master's or PhD in related field

Tools

OpenAI Evals

RAG evaluators

Weights & Biases (W&B)

PyTest

Python

SQL

Job description

Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability across multilingual and domain-specific testing.

You will design test plans, build rubrics, and document failure modes to raise standards for safety, bias checks, and prompt robustness. As a contractor, you will work remotely with flexible hours, collaborating with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose improvements to prompts, guardrails, and evaluation