Senior AI Quality Engineer — Adversarial QA for AI Systems (Hybrid)

R/GA; New York; Hybrid; GBP 101.000 - 113.000; Vollzeit

Zusammenfassung

Addition, R/GA's applied intelligence studio, seeks a Senior AI Quality Engineer to ensure high-quality AI systems through rigorous testing, planning, and red-teaming. You will collaborate with engineers, PMs, and clients to formalize quality targets and implement scalable testing practices in a hybrid NYC/Tri-State, LA, or SF Bay Area setting.

The role emphasizes testing across functional, edge, and AI-specific risks, building automated tests, logging, and evaluation frameworks to turn quality

Qualifikationen

• 4+ years of software QA, test engineering, or quality engineering experience, with at least 1–2 years working on AI-powered or LLM-based systems.
• Ability to read code and understand system architecture well enough to write meaningful test plans and probe the right seams without needing to redesign the system.
• Experience writing structured test plans and evaluation criteria, including trajectory-level assessment of multi-step reasoning and judgement.
• Hands-on fluency with AI development tools, agent frameworks, and prompt engineering, with the ability to speak the same language as engineers.
• Proficiency with modern collaboration tools (Notion, Google Workspace, Figma/FigJam, Slack).

Aufgaben

• Test Planning: translate product and technical requirements into structured test plans — covering functional behavior, edge cases, failure modes, and AI-specific risks like hallucination, prompt injection, and output drift.
• Agentic Testing: write test scripts, setup logging frameworks, analyze agent behavior and user behavior to identify failure points and opportunities for improvement.
• AI Red Teaming: systematically red-team systems before launch, probe LLМ behaviors, brand voice outputs, and multi-model pipelines for failure modes.
• Launch Gating: act as the quality gate on launches — raising blockers and documenting failure cases.
• Regression & Evaluation Frameworks: build lightweight eval frameworks and test harnesses for recurring system types to improve efficiency, speed and cost of QA and red teaming.
• Cross-Project Quality Standards: develop shared quality standards for AI systems and what 'good enough to ship' means for agentic products.
• Tracing and Documentation: document failure patterns and test results so findings point to root causes.

Kenntnisse

Test planning

AI testing

Red teaming

Automation scripting

Quality engineering

Collaboration

Tools

Notion

Google Workspace

Figma

FigJam

Slack

Jobbeschreibung

Addition, R/GA's applied intelligence studio, seeks a Senior AI Quality Engineer to ensure high-quality AI systems through rigorous testing, planning, and red-teaming. You will collaborate with engineers, PMs, and clients to formalize quality targets and implement scalable testing practices in a hybrid NYC/Tri-State, LA, or SF Bay Area setting.

The role emphasizes testing across functional, edge, and AI-specific risks, building automated tests, logging, and evaluation frameworks to turn quality