Remote AI Red Teamer: Stress-Test & Harden LLM Safety

Handshake; United States; A distancia; USD 83.000 - 152.000; Jornada completa

Descripción de la vacante

Handshake AI in the United States is seeking an AI Red Teamer (LLM Generalist) to stress-test large language models by crafting adversarial prompts and evaluating guardrails across diverse risk categories, including safety, cybersecurity, and content review.

The role involves working with multiple models via API interfaces and evaluating outputs against harm taxonomies, with a focus on responsible disclosure and reproducible experiments.

Formación

• Hands-on experience with multiple LLMs (ChatGPT, Claude, Gemini, open-source models).
• Intuition for crafting adversarial prompts; familiarity with jailbreak or evasion techniques.
• Creative, adversarial problem-solving skills.
• Clear and thoughtful written communication.
• Strong ethical judgment and ability to separate adversarial thinking from values.
• Self-directed, collaborative, and comfortable in feedback-heavy environments.
• Curiosity, persistence, and comfort with frequent experimentation.

Responsabilidades

• Craft creative prompts and multi-turn scenarios to stress-test AI guardrails across diverse risk categories.
• Discover ways around safety filters, restrictions, and defenses using jailbreak, evasion, and prompt injection techniques.
• Explore edge cases to provoke disallowed, harmful, or incorrect outputs.
• Evaluate and score model responses against structured harm taxonomies and severity rubrics.
• Document experiments clearly, including what you tried, why you tried it, and what it revealed.
• Review and refine adversarial prompts generated by other team members.
• Contribute to harm taxonomy development, calibration exercises, and inter-rater reliability work.
• Collaborate with engineers, data scientists, and researchers to share findings and strengthen defenses.
• Work with potentially disturbing content on a regular basis (see Content Warning below).
• Stay current on jailbreaks, attack methods, and evolving model behaviors.

Conocimientos

LLM Expertise

Adversarial Prompting

Ethical Judgment

Written Communication

Self-directed

Collaboration

Curiosity

Herramientas

LLM APIs

Evaluation Tooling

Python Basics

Descripción del empleo

Handshake AI in the United States is seeking an AI Red Teamer (LLM Generalist) to stress-test large language models by crafting adversarial prompts and evaluating guardrails across diverse risk categories, including safety, cybersecurity, and content review.

The role involves working with multiple models via API interfaces and evaluating outputs against harm taxonomies, with a focus on responsible disclosure and reproducible experiments.