Safety Operations Lead

Thinking Machines Lab Inc.; San Francisco (CA); Vor Ort; USD 190.000 - 300.000; Vollzeit

Benefits dieser Stelle

Health, dental & vision benefits

Unlimited PTO

Paid parental leave

Relocation support

Zusammenfassung

Thinking Machines Lab is seeking a Safety Operations Lead to fortify safety by default while enabling rapid product iteration in human-AI collaboration tools. You’ll triage cases, define policy, and build tooling with cross-functional partners to scale safe adoption.

Based in San Francisco, you’ll influence model safety, abuse detection, and account reviews, driving measurable improvements in queue throughput and risk mitigation across the product stack.

Qualifikationen

• 7+ years in an operational trust & safety, content moderation, or fraud/abuse ops role with direct, recurring responsibility for a case queue.
• Experience owning policy definition, operationalization, and enforcement end to end, evidenced by policies or programs you've built.
• Direct case experience with cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection in production.

Aufgaben

• Review flagged content, safety escalations, and account abuse signals daily. triage cases, apply policy judgment, and take action (flags, reviews, bans) continuously.
• Design and refine safety policy across the product stack with engineering, legal, safety research, and security stakeholders.
• Build and maintain tooling and automation to speed up casework (triage workflows, ban/recovery, abuse detection templates).
• Partner with product teams to embed safety into the product experience (model refusals, flagging, account review).
• Improve observability and detection for safety-relevant events to surface faster with better signal.

Kenntnisse

Trust & safety operations

Policy enforcement

Case management

AI safety familiarity

Tooling & automation

Jobbeschreibung

Thinking Machines Lab's mission is to empower humanity through advancing collaborative general intelligence. We're building a future where everyone has access to the knowledge and tools to make AI work for their unique needs and goals.

We are scientists, engineers, and builders who’ve created some of the most widely used AI products, including ChatGPT and Character.ai, open-weights models like Mistral, as well as popular open source projects like PyTorch, OpenAI Gym, Fairseq, and Segment Anything.

About the Role

We’re looking for a Safety Operations Lead who will focus on making our products safe by default while supporting fast product iteration and ambitious ideas.

You’ll work closely with product engineers, security, researchers, and designers to bake safety and integrity into the way we design, build, and ship human-AI collaboration tools. Day to day, you'll be in the moderation queue while building the tooling and policy that make the next round of moderation faster and more accurate.

What You’ll Do

• Review flagged content, safety escalations, and account-level abuse signals daily. This is a standing responsibility, not a rotation: you'll triage cases, apply policy judgment, and take action (content flags, account review, bans/recovery) on an ongoing basis.
• Use patterns from that casework to design and refine safety policy across the product stack, working with engineering, legal, safety research, and security stakeholders.
• Build and maintain tooling and automation that make your own casework faster and more consistent: triage agents, ban/recovery workflows, abuse detection frameworks and templates.
• Partner with product teams to embed safety into the product experience: model refusals, content flagging, account review, and safety protections, informed by what you're seeing in the queue.
• Improve observability and detection for safety-relevant events (model safety trends, abuse patterns, malicious behavior in production), so the next round of cases surfaces faster and with better signal.

Skills and Qualifications

• 7+ years in an operational trust & safety, content moderation, or fraud/abuse ops role with direct, recurring responsibility for a case queue.
• Experience owning policy definition, operationalization, and enforcement end to end, evidenced by specific policies or enforcement programs you've built or run.
• Direct case experience with at least one of: cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection, in a production environment.
• Working familiarity with model safety and abuse risk categories (jailbreaks, prompt injection, scaled abuse) and how to identify and mitigate them in a live product, evidenced by specific cases you've handled.
• Practical experience using AI tools (Claude, Codex, or similar) to build or accelerate operational workflows, not just as a general user of these tools.

Preferred qualifications:

We encourage you to apply even if you don’t meet all preferred qualifications.

• Experience with safety and integrity operations specifically on AI-powered products or LLM APIs, and their unique abuse patterns.
• Track record of turning recurring case patterns into reusable tooling, workflows, or process improvements, while still owning the underlying queue rather than handing it off once the interesting part is solved.
• Experience training, onboarding, or setting the quality bar for other moderators or reviewers, showing you scale a team's output rather than just your own.

You’ll Thrive in This Role if

• You have thoughtful opinions about what safe, trustworthy, frictionless user experiences look like, and you test those opinions against real cases in the queue, not just in the abstract.
• You can translate safety and technical constraints into clear product trade-offs and feature requirements, but you see that as something that grows out of daily casework, not a substitute for it.
• You bias toward speed and learning, and you measure that bias by how much faster or better the queue runs this month, not by how many new risk categories you've personally discovered.

Logistics

• Location: This role is based in San Francisco, California.
• Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $190,000 - $300,000.
• Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
• Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
• As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

Thinking Machines Lab will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of the California Fair Chance Act, the San Francisco Fair Chance Ordinance, and any other applicable state or local fair chance ordinance or law.