Head of Trust & Safety

magicschool; United States; Remote; USD 150,000 - 230,000; Full time

Benefits offered by this job

Flexible paid time off

Employer-paid health plan options

Dental and vision coverage

Equity

A 401(k) match

A monthly wellness stipend

Job summary

magicschool is seeking a Lead Trust & Safety for an AI product used in educational settings. You will own safety of model outputs, build a small team, and embed safeguards into product development and governance alongside policy, legal, and engineering colleagues.

You will establish safety criteria, lead incident response, and prepare communications for school districts and boards. Remote-friendly, full-time role with comprehensive benefits.

Qualifications

• Proven ownership of trust and safety for a consumer product at scale.
• Hands-on experience with large language model safety techniques such as evaluations, classifiers, system-level safeguards, and red-teaming.
• Experience responding to live incidents involving minors or other vulnerable users.
• Track record of building safety measures into product systems and workflows.
• Strong writing skills for safety documentation, external explanations, and incident communications.

Responsibilities

• Set policy and safeguards for AI outputs, including model evaluations, red-team exercises, monitoring, and incident response.
• Establish safety acceptance criteria early in product planning and develop evaluations and monitoring as part of product infrastructure.
• Define protections for student-facing use, including age-appropriate content, responses to self-harm or crisis situations, and escalation paths to educators.
• Lead safety incident handling and postmortems, turning findings into improvements with product and engineering teams.
• Prepare safety documentation and responses for school districts, procurement processes, and school boards; align operational practices with relevant education guidance.
• Partner with legal counsel on privacy and emerging AI requirements, and build and lead a small trust and safety team.

Skills

Trust & safety ownership

ML safety techniques

Incident response

Executive communication

Policy decisions

Safety documentation

Strong writing

Job description

Role overview

Lead trust and safety for an AI product used in educational settings, with responsibility for the safety of model outputs and the confidence of school communities. The role combines policy, technical safeguards, evaluations, incident response, and external communication, while building a small team and embedding safety into product development.

Responsibilities

• Set policy and safeguards for AI outputs, including model evaluations, red-team exercises, monitoring, and incident response.
• Establish safety acceptance criteria early in product planning and develop evaluations and monitoring as part of product infrastructure.
• Define protections for student-facing use, including age-appropriate content, responses to self-harm or crisis situations, and escalation paths to educators.
• Lead safety incident handling and postmortems, turning findings into improvements with product and engineering teams.
• Prepare safety documentation and responses for school districts, procurement processes, and school boards; align operational practices with relevant education guidance.
• Partner with legal counsel on privacy and emerging AI requirements, and build and lead a small trust and safety team.

Requirements

• Proven ownership of trust and safety or responsible AI for a consumer or youth-facing product at scale, including real incidents and policy decisions.
• Hands-on experience with large language model safety techniques such as evaluations, classifiers, system-level safeguards, and red-teaming.
• Experience responding to live incidents involving minors or other vulnerable users.
• Track record of building safety measures into product systems and workflows.
• Strong judgment under uncertainty and clear communication of safety decisions to executive and district audiences.
• Strong writing skills for safety documentation, external explanations, and incident communications.

Nice to have

• Experience in K-12 or youth-focused products, familiarity with education AI frameworks or district procurement, or work with child-safety organizations.
• Data or machine-learning fluency and comfort representing safety in public or district settings.

Benefits and work setup

The role supports working from home.

Full-time benefits include flexible paid time off, employer-paid health plan options, dental and vision coverage, equity, a 401(k) match, and a monthly wellness stipend.

• Flexible paid time off
• Employer-paid health plan options
• Dental and vision coverage
• Equity
• A 401(k) match
• A monthly wellness stipend