Senior QA Analyst / Tester

Orion Innovation; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Orion Innovation is seeking a high-caliber AI Red Team Tester to conduct adversarial testing of AI models, evaluate security tools, and drive remediation strategies. The role focuses on real-world deployment scenarios, bias detection, and safeguarding against harmful outputs while collaborating with global security teams and stakeholders.

The ideal candidate has at least five years in penetration testing or AI red teaming, a strong CS/IT background, and excellent communication skills to

Qualifications

• Minimum 5 years of penetration testing, red team operations, or AI Red Team experience.
• Background in AI red teaming or offensive cyber operations.
• Understanding of AI, machine learning, software applications, cloud computing, and networking.
• Excellent communication and stakeholder management skills.
• Ability to work effectively across geographies and time zones.

Responsibilities

• Design and execute adversarial testing campaigns against AI models and systems.
• Research attack vectors and prompt injection techniques to identify vulnerabilities and unintended behaviors.
• Evaluate and introduce AI security tools to reduce mean time to detect and respond to AI threats.
• Conduct red team exercises simulating real-world deployment scenarios and probe for biases and harmful outputs.
• Create adversarial datasets and benchmarks to evaluate model robustness.
• Collaborate with security teams to document findings, vulnerabilities, and remediation recommendations.
• Develop and maintain process documentation and reports on testing activities.

Skills

Penetration testing

Red team operations

AI Red Team experience

Communication skills

Stakeholder management

Education

Bachelor's degree in CS/IT/AI/Cybersecurity

Tools

SPLX

Garak

TextAttack

PyRIT

Burp Suite

Metasploit

Nessus

Cobalt Strike/Mythic/C2

NMAP

sqlmap

Job description

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

The Red Team Tester will conduct testing of AI models to find vulnerabilities, develop new evaluation frameworks, communicate risks to stakeholders across the global teams and the network of member firms, and collaborate with defensive and development teams.

Key accountabilities % of time Accountability

• 10 Design and execute comprehensive adversarial testing campaigns against AI models, including but not limited to large language models, multimodal systems, and autonomous agents.
• 10 Research attack vectors and prompt injection techniques to identify model vulnerabilities, jailbreaks, and unintended behaviors.
• Evaluate and introduce AI Security tools that decrease mean time to detect and respond to AI-specific threats.
• 50 Conduct red team exercises simulating real-world deployment scenarios and edge cases. Systematically probe for bias, toxicity, misinformation generation, and other harmful outputs across diverse contexts and demographics. Improve firm’s security posture against emerging AI threats. Concentration is on executing standardized security tests.
• 10 Create adversarial datasets and benchmarks to evaluate model robustness under various attack conditions.
• 10 Collaborate with security teams (e.g., GSOC, member firm security teams, developers) to perform red teaming activities, and document and present findings, vulnerabilities, and remediation recommendations to drive the mitigation of identified risks.
• 10 Develop and maintain process documentation, create reports on testing activities, track operational metrics, and perform other programmatic tasks as required to support the AI Red Team's function.

This is the job for you if you have… Minimum 5 years of penetration testing, red team operations, or AI Red Team experience. Computer science, information technology, programming, development, Artificial Intelligence, or cybersecurity degree (Bachelor's or higher preferred) from an accredited college or university or equivalent work experience. Background in AI red teaming, red teaming, or offensive cyber operations. Understanding of Artificial Intelligence, Machine Learning, software applications, cloud computing, and networking. Excellent communication and stakeholder management skills. Ability to work effectively across geographies and time zones.

Core technical skills include:

• Testing tools: SPLX, Garak, TextAttack, PyRIT, Burp Suite, Metasploit, Nessus, Cobalt Strike/Mythic/C2, NMAP, sqlmap, or similar tools
• Web application technologies and layer 7 protocols (HTTP, DNS, FTP)
• Technical knowledge of Generative AI, Agentic AI, A2A, and/or MCP
• Programming languages: Python, Ruby, Go, C++, PowerShell, bash or similar
• Certifications: OSCP, GPEN, GPXN, AI Red Teamer Job Role Path (HackTheBox), AISEC+, GASAE, GOAA, GASAE, OSAI+, or other industry AI or Red Team related certifications desired

Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains: What information we collect during our application and recruitment process and why we collect it; How we handle that information; and How to access and update that information. Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.