Senior QA Engineer: AI-Driven Testing & Benchmarking

Cisco Systems, Inc; New York (NY); On-site; USD 168,000 - 245,000; Full time

Job summary

Cisco Systems, Inc in New York seeks a senior Quality Engineer in the Enterprise Security Apps team to translate complex quality challenges into scalable testing infrastructure for AI-powered security solutions.

You will design benchmarking protocols, implement AI-driven tests, and collaborate on reviews to ensure reliability, security, and performance across Cisco's unified security ecosystems. You will also drive experimentation with POCs and LLM-based validation strategies.

Qualifications

• Bachelor's degree plus 8+ years of experience in software development and test automation, including experience building large-scale testing ecosystems.
• Programming experience with Python and Groovy, involving the design of scalable, maintainable, and high-performance testing infrastructure.
• Experience working with security compliance frameworks.
• Experience implementing modern performance testing methodologies.
• Experience using AI to build automation, including writing AI skills to build test automation.

Responsibilities

• Contribute to the development of foundations and evaluation frameworks, ensuring assessment capabilities for AI agents, workflows, and system processes.
• Develop AI-based solutions or agents to mitigate risk and enhance system integrity across testing methodologies.
• Participate in code and design reviews, support technical collaboration, and assist in the validation of technical hypotheses.
• Implement the lifecycle of proof-of-concepts (POCs) while adhering to standards for experimentation and the validation of testing paradigms.
• Execute advanced validation techniques, including LLM-as-judge methodologies, tenant isolation probes, and behavioral regression suites.

Skills

Python

Groovy

AI testing

Test automation

Security compliance

Education

Bachelor's degree in Computer Science or related field

Tools

Playwright

OCSF

Caldera

vacuum linter

tenant isolation probes

audit log validation

Job description

Cisco Systems, Inc in New York seeks a senior Quality Engineer in the Enterprise Security Apps team to translate complex quality challenges into scalable testing infrastructure for AI-powered security solutions.

You will design benchmarking protocols, implement AI-driven tests, and collaborate on reviews to ensure reliability, security, and performance across Cisco's unified security ecosystems. You will also drive experimentation with POCs and LLM-based validation strategies.