Safety Operations Lead

Thinking Machines Lab; San Francisco (CA); On-site; USD 190,000 - 300,000; Full time

Benefits offered by this job

Health benefits

Dental benefits

Vision benefits

Unlimited PTO

Parental leave

Relocation support

Job summary

Thinking Machines Lab is seeking a Safety Operations Lead to embed safety by default while enabling fast product iteration. You will triage content flags, design safety policy, and build tooling to accelerate casework, partnering with product, security, and legal teams.

You’ll influence product safety across the stack, from model refusals to abuse detection, ensuring scalable protections in a cutting-edge AI environment.

Qualifications

• 7+ years in operational trust & safety, content moderation, or fraud/abuse ops with direct, recurring case-queue responsibility.
• Experience owning policy definition, operationalization, and enforcement end-to-end.
• Direct case experience with cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection in production.
• Familiarity with model safety and abuse risk categories and mitigation in live products.
• Experience using AI tools (Claude, Codex, or similar) to build or accelerate operational workflows.

Responsibilities

• Review flagged content, safety escalations, and account-level abuse signals daily; triage cases, apply policy judgment, and take action (flags, bans) ongoing.
• Design and refine safety policy across the product stack based on casework, collaborating with engineering, legal, safety research, and security.
• Build and maintain tooling and automation to speed up casework and ensure consistency (triage workflows, ban/recovery).
• Partner with product teams to embed safety into the product experience (model refusals, content flagging, account review, protections).
• Improve observability and detection for safety-relevant events so the next round of cases surfaces faster with better signal.

Skills

Trust & safety operations

Policy ownership

Cybersecurity abuse

Model safety awareness

AI tooling

Job description

Thinking Machines Lab's mission is to empower humanity through advancing collaborative general intelligence. We're building a future where everyone has access to the knowledge and tools to make AI work for their unique needs and goals.

We are scientists, engineers, and builders who’ve created some of the most widely used AI products, including ChatGPT and Character.ai, open-weights models like Mistral, as well as popular open source projects like PyTorch, OpenAI Gym, Fairseq, and Segment Anything.

About the Role

We’re looking for a Safety Operations Lead who will focus on making our products safe by default while supporting fast product iteration and ambitious ideas.

You’ll work closely with product engineers, security, researchers, and designers to bake safety and integrity into the way we design, build, and ship human-AI collaboration tools. Day to day, you'll be in the moderation queue while building the tooling and policy that make the next round of moderation faster and more accurate.

What You’ll Do

• Review flagged content, safety escalations, and account-level abuse signals daily. This is a standing responsibility, not a rotation: you'll triage cases, apply policy judgment, and take action (content flags, account review, bans/recovery) on an ongoing basis.
• Use patterns from that casework to design and refine safety policy across the product stack, working with engineering, legal, safety research, and security stakeholders.
• Build and maintain tooling and automation that make your own casework faster and more consistent: triage agents, ban/recovery workflows, abuse detection frameworks and templates.
• Partner with product teams to embed safety into the product experience: model refusals, content flagging, account review, and safety protections, informed by what you're seeing in the queue.
• Improve observability and detection for safety-relevant events (model safety trends, abuse patterns, malicious behavior in production), so the next round of cases surfaces faster and with better signal.

Skills and Qualifications

• 7+ years in an operational trust & safety, content moderation, or fraud/abuse ops role with direct, recurring responsibility for a case queue.
• Experience owning policy definition, operationalization, and enforcement end to end, evidenced by specific policies or enforcement programs you've built or run.
• Direct case experience with at least one of: cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection, in a production environment.
• Working familiarity with model safety and abuse risk categories (jailbreaks, prompt injection, scaled abuse) and how to identify and mitigate them in a live product, evidenced by specific cases you've handled.
• Practical experience using AI tools (Claude, Codex, or similar) to build or accelerate operational workflows, not just as a general user of these tools.

Preferred qualifications:

• Experience with safety and integrity operations specifically on AI-powered products or LLM APIs, and their unique abuse patterns.
• Track record of turning recurring case patterns into reusable tooling, workflows, or process improvements, while still owning the underlying queue rather than handing it off once the interesting part is solved.
• Experience training, onboarding, or setting the quality bar for other moderators or reviewers, showing you scale a team's output rather than just your own.

You’ll Thrive in This Role if

• You have thoughtful opinions about what safe, trustworthy, frictionless user experiences look like, and you test those opinions against real cases in the queue, not just in the abstract.
• You can translate safety and technical constraints into clear product trade-offs and feature requirements, but you see that as something that grows out of daily casework, not a substitute for it.
• You bias toward speed and learning, and you measure that bias by how much faster or better the queue runs this month, not by how many new risk categories you've personally discovered.

Logistics

• Location: This role is based in San Francisco, California.
• Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $190,000 - $300,000.
• Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
• Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
• As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

Thinking Machines Lab will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of the California Fair Chance Act, the San Francisco Fair Chance Ordinance, and any other applicable state or local fair chance ordinance or law.