Head of Trust & Safety

OpenKrill; United States; Remote; USD 170,000 - 230,000; Full time

Benefits offered by this job

Unlimited time off

Health insurance

Stock options

401k match

Job summary

MagicSchool is seeking a senior Safety/Responsible AI leader to own trust and safety for a youth-facing AI platform. You will manage policy, guardrails, and incident response, while aligning with districts and regulators.

The role includes building a small safety team and scaling safety infrastructure to enable fast, safe shipping.

Qualifications

• Must have owned trust & safety or responsible AI for a consumer or youth-facing product at scale.
• Hands-on with LLM safety tooling: evals, classifiers, guardrails, red-teaming.
• Safety as velocity enabler; build infrastructure so teams ship confidently.
• Communicate safety tradeoffs to executives and districts clearly.
• Strong writing for safety docs, explainers, and incident communications.
• Ability to act quickly on ambiguous situations and public stakes.

Responsibilities

• Own the safety of AI output to teachers and students: policy, guardrails, evals, red-teaming, incident response.
• Own trust posture with districts: safety docs, RFP responses, state guidance alignment.
• Develop safety evaluations and monitoring as core product infrastructure.
• Own student-facing safety: age-appropriate content, crisis handling, educator escalation.
• Run safety incident process end-to-end and close the postmortem loop into product.
• Partner with Legal on COPPA, FERPA, and evolving AI regulation; define operational reality.
• Build and lead a small team (1–3 to start) and scale the function through 2027.

Skills

Trust & Safety ownership

LLM safety tooling

Safety as product infrastructure

Ambiguity-driven decisions

Technical writing

Hypothesis-driven testing

Tools

OpenAI

Anthropic

Google Vertex

Moderation APIs

Datadog

Linear

Slack

Notion

Job description

WHO WE ARE: MagicSchool is the premier generative AI platform for teachers. We're just over 2 years old, and more than 7 million teachers from all over the world have joined our platform. Join a top team at a fast growing company that is working towards real social impact. Make an account and try us out at our website https://www.magicschool.ai and connect with our passionate community on our Wall of Love https://love.magicschool.ai/all.

RESPONSIBILITIES

• Own the safety of AI output to teachers and students: policy, model-level guardrails, evals, red-teaming, and incident response.
• Own the trust posture with districts: safety documentation, RFP and school-board responses, and alignment with state AI-in-education guidance.
• Develop safety evaluations and monitoring as integral components of product infrastructure; ensure that each new tool includes safety acceptance criteria established during the scoping phase, rather than at launch.
• Own student-facing safety specifically: age-appropriate content, self-harm and crisis handling, and escalation paths to educators.
• Run the safety incident process end to end and own the postmortem loop back into product and engineering.
• Partner with Legal on COPPA, FERPA, state privacy laws, and emerging AI regulation. T&S owns the operational reality; Legal owns the legal position.
• Build and lead a small team (1–3 to start), scaling the function through 2027.

QUALIFICATIONS / COMPETENCIES / SKILLS

Must Haves

• Has owned trust & safety or responsible AI for a consumer or youth-facing product at scale: real incidents, real policy decisions, real regulators.
• Hands-on with LLM safety: evals, classifiers, prompt/system-level guardrails, red-teaming. Can read an eval result and say what to change.
• Has made safety a velocity enabler, not friction — builds guardrails as infrastructure so teams ship faster with confidence.
• Makes judgment calls under ambiguity and owns them publicly, to districts and to the exec team.
• Strong writer: safety docs, district-facing explainers, incident communications.
• Operates hypothesis-first and escalates one-way doors fast rather than sitting on them.

Nice to Haves

• K-12 or youth-platform experience (GoGuardian, Securly, Roblox, Discord, YouTube Kids, Epic Games).
• Familiarity with state AI-in-education frameworks and district procurement questionnaires.
• Prior work with child-safety NGOs or standards bodies (Common Sense Media, NCMEC, Thorn, ISTE).
• Data/ML fluency to partner directly with engineering on classifiers and eval pipelines.
• Comfortable being the external face of safety at conferences, with press, and in front of district boards.

REQUIRED EXPERIENCE

• Proven ownership of trust & safety or responsible AI at a consumer or youth-facing product at scale.
• Hands-on experience with LLM safety tooling: evals, classifiers, and red-teaming.
• Experience handling live incidents involving minors or vulnerable users.
• Track record of building safety as product infrastructure, not a gate.
• Demonstrated ability to communicate safety tradeoffs clearly to executive and district-level audiences.

TECH STACK

• LLM layer: multi-provider (OpenAI / Anthropic / Google via GCP Vertex); moderation APIs and custom classifiers.
• Eval and red-team tooling (to be built out); Datadog for output observability and alerting.
• Product stack context: TypeScript / Next.js on Vercel, Supabase (PostgreSQL) backend.
• Incident and knowledge tooling: Linear, Slack workflows, Notion.

Why Join Us?

• Work on cutting-edge AI technology that directly impacts educators and students.
• Join a mission-driven team passionate about making education more efficient and equitable.
• Flexibility of working from home, while fostering a unique culture built on relationships, trust, communication, and collaboration with our team - no matter where they live.

For full time employees:

• Unlimited time off to empower our employees to manage their work-life balance. We work hard for our teachers and users, and encourage our employees to rest and take the time they need.
• Choice of employer-paid health insurance plans so that you can take care of yourself and your family. Dental and vision are also offered at very low premiums.
• Every employee is offered generous stock options, vested over 4 years.
• 401k match & monthly wellness stipend.

Our Values:

• Educators are Magic: Educators are the most important ingredient in the educational process - they are the magic, not the AI. Trust them, empower them, and put them at the center of leading change in service of students and families.
• Joy and Magic: Bring joy and magic into every learning experience - push the boundaries of what’s possible with AI.
• Community: Foster community that supports one another during a time of rapid technological change. Listen to them and serve their needs.
• Innovation: The education system is outdated and in need of innovation and change - AI is an opportunity to bring equity, access, and serve the individual needs of students better than we ever have before.
• Responsibility: Put responsibility and safety at the forefront of the technological change that AI is bringing to education.
• Diversity: Diversity of thought, perspectives, and backgrounds helps us serve the wide audience of educators and students around the world.
• Excellence: Educators and students deserve the best - and we strive for the highest quality in everything we do.