Principal Software QA Specialist

Fortinet; Burnaby; On-site; CAD 80,000 - 100,000; Full time

Job summary

A leading company in cybersecurity is seeking a talented individual to design and execute test cases for AI security firewall features. The role involves building test datasets, validating model integration, and collaborating with R&D teams. Candidates should have experience with containerized deployment and stay updated on industry standards.

Responsibilities

• Design and execute system-level test cases for AI security firewall features.
• Build and manage test datasets for prompt scenarios.
• Collaborate with teams to develop verifiable test scenarios.

Tools

Docker

Kubernetes

Job description

Job Responsibilities:

1. Design and execute system-level and module-level test cases to ensure the correctness and performance of AI security firewall features, including prompt filtering, content moderation, and response policy enforcement.
2. Build and manage test datasets to simulate various prompt scenarios; evaluate the model’s ability to detect and defend against attacks such as prompt injection or misuse.
3. Validate the effectiveness of model integration by measuring metrics like detection accuracy, false positive rate (FPR), and false negative rate (FNR), while improving test coverage.
4. Collaborate with R&D, security, and product teams to develop verifiable test scenarios based on AI security requirements; assist in bug reproduction and root cause analysis.
5. Set up test environments and perform regression, performance, and stress testing regularly.
6. Contribute to developing hybrid local/cloud testing platforms; demonstrate proficiency in containerized deployment (Docker/Kubernetes) for flexible testing across different model versions and security policies.
7. Stay informed about industry standards and best practices in LLM/Generative AI security (e.g., OWASP Top 10 for LLMs), and integrate relevant threat simulations into test plans.