Senior AI Quality Engineer

rga; Los Angeles (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

Addition, R/GA's applied intelligence studio, seeks a Senior AI Quality Engineer to ensure the quality of AI systems through rigorous testing, red-teaming, and automation. You’ll write test plans, automate testing, and align with engineers and PMs on ship criteria to tighten quality across deployments.

You’ll help evolve quality into a measurable discipline, design adversarial prompts and logging, and collaborate with cross-functional teams to scale testing practices for complex AI products.

Qualifications

• Experience designing structured test plans for AI systems.
• Proven ability to perform adversarial testing and red-teaming.
• Experience building lightweight eval frameworks for QA.

Responsibilities

• Translate product and technical requirements into structured test plans.
• Write test scripts and set up logging to analyze AI system behavior.
• Systematically red-team AI systems before launch and document failures.
• Maintain lightweight evaluation frameworks for recurring system types.
• Collaborate with engineers and PMs to raise blockers and ship criteria.

Skills

Test Planning

Red Teaming

Automation

Logging & Metrics

Tools

Claude Code

QA Automation

Logging Frameworks

Job description

About Addition

Addition is R/GA's applied intelligence studio. We work with many of the world's leading brands to solve problems at the intersection of frontier AI and human-centered design. We collaborate with our clients to implement and scale AI systems that create real-world value-improving communications, customer experiences, and internal business workflows.

About R/GA

In our 40-year history, we've changed a lot-from a culture-defining film title studio (Superman, Alien, Ghostbusters); to an interactive advertising agency; to an award-winning product and marketing agency with design, technology, communications, ventures and IP practices. We like change, and like to navigate our clients through it too. Our approach to work is always human first. This unwavering focus helps us create meaningful and memorable experiences that live across digital platforms and often blur the line between the digital and real worlds. We believe disruptive thinking happens when you collide experts in design, technology and culture.

About the Role

As our Senior AI Quality Engineer, your job is to ensure that the solutions we ship are high quality by methodically testing them and working with engineering to implement improvements. This requires a test and verify mindset; you don't assume things are working, you understand the shape of the system and test it from every angle to ensure quality. During planning phases, you explore system architecture, write test plans, and align with the team on what "quality" looks like. As systems are built, you red-team them, code scripts to automate and scale testing processes, implement logging, and methodically analyze the results. You think like an AI agent, stress-testing the behavior of AI systems under a wide range of conditions beyond the happy-path.

This is a new role at Addition, created because the complexity of the AI systems we ship has outgrown traditional QA practices and ad-hoc evaluation. We need someone who turns quality into a new discipline for the AI systems era - evolving fuzzy notions of "quality" into measurable signals, and bringing the rigor of a structured adversarial mindset. This role sits within our engineering team, working closely with engineers, PMs, cross-functional project teams, and clients.

Primary Responsibilities

• Test Planning: During project planning phases, translate product and technical requirements into structured test plans - covering functional behavior, edge cases, failure modes, and AI-specific risks like hallucination, prompt injection, and output drift.


• Agentic Testing: Working with coding agents like Claude Code and local or online versions of AI System builds to write test scripts, setup logging frameworks, analyze agent behavior, user behavior, to identify points of failure and opportunities for improvement. This is automated and streamlined for scale, but always done with intentionality, human supervision, and verification along the way.


• AI Red Teaming: When systems are built, systematically red-team them before launch. Probe LLM behaviors, agent decision chains, brand voice outputs, and multi-model pipelines for failure modes that functional testing won't surface. Assess the agent's overall trajectory, tool selection and reasoning steps. Design prompts that attempt to manipulate behavior, discover conflicting instructions and surface failure cases before they reach production.


• Launch Gating: Act as the quality gate on launches - raising blockers, documenting failure cases, and holding the line on ship criteria. You are the friction layer between "built" and "live."


• Regression & Evaluation Frameworks: Build and maintain lightweight eval frameworks and test harnesses for recurring system types (e.g., brand voice agents, agentic creative tools, generative UIs) to systematize and improve efficiency, speed and cost of QA and red teaming practices over time.


• Cross-Project Quality Standards: Work across our project portfolio to develop shared quality standards for AI systems - what "good enough to ship" actually means for agentic products.


• Tracing and Documentation: Document failure patterns, red team findings, and test results in a form that engineers and PMs can act on. When something fails, trace tool interactions across the reasoning steps to pinpoint how the input propagated through the agent's logic - so a finding points at a root cause, not just a symptom.

The Ideal Person

• Problem Solver & Adaptable: Thrives in fast-paced, high-ambiguity environments and turns complex challenges into structured, elegant solutions. Brings a genuinely adversarial mindset-finding failure interesting, not frustrating.


• High EQ & Collaborative: A cross-functional team player who builds strong, trusting relationships with internal teams, C-suite executives, and enterprise clients.


• Authentic Communicator: Exceptional written and verbal communication skills, able to translate complex technical/AI concepts into clear, actionable, and persuasive id