A complete application in a minute
Tailored resume and cover letter, ready to send.
Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows.
You will partner with product and engineering for new AI releases, define quality metrics, and stay current on evaluation techniques. This role offers a fast-paced, high-impact environment with growth opportunities.
At Rootly, we are on a mission to be the go-to way companies respond when things go wrong, helping every organization be more reliable. We do this by building an industry-leading incident management platform that allows companies around the world to consistently and quickly resolve incidents. We are not simply transforming an industry, we are carving out a new multi-billion-dollar category ourselves and need incredible talent to achieve this ambitious goal together.
Customers love Rootly. Some of the fastest growing companies around the world such as NVIDIA, Figma, Canva, Tripadvisor, Squarespace and more rely on Rootly to power their critical incident management process. They obsess over our delightful enterprise-ready platform and unique partnership model. See why our customers have reviewed us 5 stars on G2.
Investors love Rootly. We are backed by some of the most respected funds in the world from Y Combinator to operators like the CTO of Dropbox and GitHub. We'd be happy to disclose our entire funding and profitability picture live during the interview. As a culture we relentlessly put transparency first. We conduct monthly financial reviews as a team so everyone has a pulse on the health of the business and publish what we are building in our weekly changelog.
Rootly is building the AI-native future of incident management, and we need someone who can push our AI to its limits before our customers do. As our AI Quality Engineer, you'll own the evaluation and optimization of Rootly's agentic AI features -- designing test scenarios, running adversarial prompts, interpreting outputs, and working directly with engineering and product to close the loop on performance.
This isn't traditional QA. You'll spend your days thinking like an attacker, a confused user, and a power user all at once -- probing how our AI agents reason, make decisions, and handle edge cases across complex incident workflows.
We're not just another startup. We're building something category-defining and want teammates who crave ownership, love solving hard problems, and thrive in a high‑bar, high-impact environment.
Here's what you can expect when you join Rootly:
This is a newly created vacancy.
As we build out the team, we may use AI to support parts of our interview process, like reviewing assessments, but it's a tool that helps our people, not a substitute for their judgement.
Accommodations are available on request at any stage of our recruitment process. Contact recruitment@rootly.com and we will work with you to meet your needs confidentially.
Rootly is an equal opportunity employer. We do not discriminate on the basis of race, ancestry, place of origin, colour, ethnic origin, citizenship, creed, sex, sexual orientation, gender identity, gender expression, age, record of offences, marital status, family status, disability, or any other ground protected by the Ontario Human Rights Code or applicable law.