Senior AI Red Team Scientist for Safety & Evaluation

SupportFinity™; New York (NY); On-site; USD 190,000 - 211,000; Full time

Benefits offered by this job

401(k) plan

Bonus program

Equity opportunity

Job summary

Uber is seeking a Senior Applied Scientist to join the AI Red Teaming efforts, focusing on adversarial evaluation, failure analysis, and risk discovery in AI models and agents.

You will design experiments and evaluation frameworks to surface unsafe behaviors, including prompt injection, jailbreaking, and memory poisoning. This role emphasizes safety, robustness, and rigorous scientific methods in real-world AI systems.

Qualifications

• 5+ years of experience as a Data Scientist, Applied Scientist, or ML Scientist.
• Hands-on experience with LLMs or generative AI systems.
• Direct experience with AI red teaming, model safety, or adversarial evaluation.
• Direct experience with prompt injection, jailbreaks, and LLM failure modes.
• Strong background in experimental design, evaluation, and statistical analysis.
• Experience analyzing complex model behavior and failure cases beyond standard metrics.
• Proficiency in Python and common DS/ML tooling.

Responsibilities

• Design and execute AI red-teaming experiments against LLMs and AI agents to identify: prompt injection, jailbreaking, policy bypass, model and tool poisoning, context/memory poisoning, behavioral drift, unsafe autonomy.
• Develop adversarial datasets, probes, and test harnesses to systematically evaluate model and agent behavior under attack.
• Define and track AI risk metrics beyond accuracy (e.g., failure rates, drift indicators, unsafe action likelihood, confidence miscalibration).
• Analyze agent workflows and decision traces to understand how failures emerge across multi-step reasoning and tool use.
• Collaborate with security engineers and AI platform teams to translate findings into guardrails, mitigations, and design improvements.
• Build reusable evaluation pipelines to support continuous red teaming and regression testing as models and agents evolve.

Skills

5+ years experience

LLMs / Generative AI

AI red teaming

Prompt injection / jailbreaks

Experimental design

Python

Job description

Uber is seeking a Senior Applied Scientist to join the AI Red Teaming efforts, focusing on adversarial evaluation, failure analysis, and risk discovery in AI models and agents.

You will design experiments and evaluation frameworks to surface unsafe behaviors, including prompt injection, jailbreaking, and memory poisoning. This role emphasizes safety, robustness, and rigorous scientific methods in real-world AI systems.