Remote AI QA Trainer for Model Safety & Quality

agency; Stati Uniti; Remoto; USD 8300 - 90.000; Tempo pieno

Vantaggi offerti da questo lavoro

Remote contract

Flexible hourly rate

Descrizione del lavoro

agency is seeking a contract QA specialist to assess the accuracy, safety, and reliability of language models through structured testing and failure analysis. This remote, contract role focuses on evaluation design, adversarial testing, and quality reporting for AI/ML systems.

You will design test plans, run regression checks, document reproducible failures, and report findings with clear root-cause hypotheses. The hourly pay ranges from $6 to $65, depending on experience and location.

Competenze

• Strong understanding of AI/ML quality assurance and model evaluation.
• Clear, evidence-based communication and issue reproduction capabilities.
• Degree in CS, data science, computational linguistics, statistics, or related field preferred.
• Experience shipping QA for AI/ML systems and designing rubrics is valuable.
• Python or SQL test automation experience; familiarity with PyTest or evaluation frameworks helpful.
• Knowledge of grounding checks, fairness audits, regression testing, and retrieval-augmented systems beneficial.

Mansioni

• Test model responses against realistic prompts and scenarios for accuracy and grounding.
• Design test plans, regression suites, scoring rubrics, and pass/fail criteria.
• Evaluate risks such as hallucinations, jailbreaks, bias, unsafe outputs, and retrieval errors.
• Document reproducible failures and propose root-cause hypotheses; report findings clearly.
• Suggest improvements to prompts, safeguards, and evaluation measures like precision and latency.
• Contribute to red-team exercises, test automation, and dashboards tracking quality over time.

Conoscenze

Quality assurance AI/ML

Clear communication

QA for AI/ML systems

Python/SQL test automation

PyTest / evaluation frameworks

Grounding checks & fairness audits

Formazione

CS/DS/Computational Linguistics degree

Strumenti

PyTest

LLM evaluation frameworks

Retrieval-augmented systems knowledge

Descrizione del lavoro

agency is seeking a contract QA specialist to assess the accuracy, safety, and reliability of language models through structured testing and failure analysis. This remote, contract role focuses on evaluation design, adversarial testing, and quality reporting for AI/ML systems.

You will design test plans, run regression checks, document reproducible failures, and report findings with clear root-cause hypotheses. The hourly pay ranges from $6 to $65, depending on experience and location.