Staff Software Engineer, AI Safety & Abuse Detection

EngineersOfAI; San Francisco, Northern (CA, KY); Hybrid; USD 320,000 - 485,000; Full time

Job summary

Anthropic in San Francisco is seeking a software engineer for the Safeguards team to build safety and oversight mechanisms for our AI systems. You will monitor models, prevent misuse, and ensure user wellbeing across the stack.

This role focuses on detecting unwanted model behaviors, enforcing policies, and surfacing abuse patterns to researchers for hardening models at the training stage. You will contribute to scalable defenses and real-time safety improvements.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering or comparable experience.
• Proficiency in Python and Typescript.
• Ability to work across the stack.
• Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders.

Responsibilities

• Develop monitoring systems to detect unwanted behaviors from API partners and surface results in dashboards for analysts.
• Build abuse detection mechanisms and infrastructure.
• Surface abuse patterns to research teams to harden models at the training stage.
• Build multi-layered defenses for real-time safety improvements at scale.

Skills

Python

Typescript

Cross-stack development

Communication

Education

Bachelor's degree in Computer Science or related field

Job description

Anthropic in San Francisco is seeking a software engineer for the Safeguards team to build safety and oversight mechanisms for our AI systems. You will monitor models, prevent misuse, and ensure user wellbeing across the stack.

This role focuses on detecting unwanted model behaviors, enforcing policies, and surfacing abuse patterns to researchers for hardening models at the training stage. You will contribute to scalable defenses and real-time safety improvements.