Senior AI Agent Quality Engineer

Adobe; New York (NY); On-site; USD 153,000 - 222,000; Full time

Job summary

Adobe is seeking a Quality Engineer for the Agentic Harness team in New York, focusing on validating agent behavior, tool use, and outputs. You will design, execute, and maintain automated test suites while performing manual testing to uncover edge cases before production releases.

Join a fast-moving, evolving AI/LLM systems environment, collaborating with engineers and data scientists to strengthen test coverage and reliability.

Qualifications

• Bachelor’s degree in Computer Science or a related field, or equivalent experience.
• Several years of experience in quality engineering, test automation, or QA, ideally on complex, evolving software.
• Hands‑on experience writing automated tests and test frameworks, with solid test design fundamentals.
• Comfort digging into bugs and failures across a stack to identify root cause, not just symptoms.
• Familiarity with testing LLM‑based or agentic systems and how their behavior differs from traditional software.
• Ability to operate in a fast-moving, ambiguous environment with evolving technology and user expectations.

Responsibilities

• Design, write, and maintain automated test suites and test plans covering the agentic harness’s behavior, tool use, and outputs.
• Implement manual and exploratory testing on new agent capabilities, identifying edge cases and failure modes before they reach production.
• Investigate test failures and reported bugs to isolate root cause, working with engineers to reproduce, document, and resolve issues.
• Partner with the Agent Evals team to close coverage gaps and improve the reliability of quality signals.

Skills

Quality engineering

Test automation

LLM testing

Bug investigation

Edge cases

Education

Bachelor's degree in CS or related

Tools

Automation frameworks

Job description

Adobe is seeking a Quality Engineer for the Agentic Harness team in New York, focusing on validating agent behavior, tool use, and outputs. You will design, execute, and maintain automated test suites while performing manual testing to uncover edge cases before production releases.

Join a fast-moving, evolving AI/LLM systems environment, collaborating with engineers and data scientists to strengthen test coverage and reliability.