Lead Researcher, AI Security Benchmarks

Vals AI, Inc.; San Francisco (CA); Vor Ort; USD 150.000 - 230.000; Vollzeit

Benefits dieser Stelle

Relocation support

Health/dental insurance

Lunch and dinner provided

Unlimited PTO

$1,500 housing stipend (within one‑m​i

Housing stipend within 1 mile radius

Collaborate with leading AI labs

Opportunity to publish

Zusammenfassung

Vals AI, Inc. seeks an exceptional researcher to design and build the next generation of AI benchmarks for cybersecurity. You will create high-impact evaluations that push what we can measure in foundation models, shaping how the world evaluates AI systems.

The role offers relocation and in-person collaboration in San Francisco with leading labs and major institutions. You will lead design and development of novel benchmarks, collaborate with labs and security teams, and publish findings to

Qualifikationen

• Hands-on security experience (e.g., security engineer, pentester, red teamer, or vulnerability researcher).
• Strong grounding in operating systems, cloud infrastructure and networking.
• Interest in benchmarking and prior cyber evals, CTFs or security benchmarks preferred.
• AI-forward: use LLMs in security work.
• Clear communication of complex research ideas to technical and non-technical audiences.
• Experience collaborating in research teams and integrating feedback.
• Location: in-person SF team with relocation/transport support.

Aufgaben

• Design and develop novel, high-impact benchmarks for cybersecurity real-world capabilities.
• Conduct research to ensure benchmarks are valid, reliable, and meaningful.
• Collaborate with foundation model labs and enterprise security teams to understand evolving evaluation needs.
• Analyze model performance across benchmarks and articulate results.
• Publish research findings and contribute to security and evaluation communities.
• Work with the infrastructure team to implement benchmark designs at scale.
• Stay updated with latest developments in LLM cyber capabilities and evaluation methodologies.

Kenntnisse

Cybersecurity

Systems depth

Benchmarking

AI-forward

Communication

Collaboration

San Francisco location

Tools

Python

Django

AWS

CDK

React

Jobbeschreibung

Vals AI, Inc. seeks an exceptional researcher to design and build the next generation of AI benchmarks for cybersecurity. You will create high-impact evaluations that push what we can measure in foundation models, shaping how the world evaluates AI systems.

The role offers relocation and in-person collaboration in San Francisco with leading labs and major institutions. You will lead design and development of novel benchmarks, collaborate with labs and security teams, and publish findings to