Founding Research Engineer

Raindrop; San Francisco (CA); On-site; USD 130,000 - 210,000; Full time

Job summary

Raindrop is seeking a founding Research Engineer to study how AI agents behave in production. You’ll join an early team, set the research agenda, and publish what we learn.

Your focus includes agent behavior, simulations, detection of unseen issues, and high-impact open questions. You will design evaluations, publish findings, and prototype methods for product integration.

Qualifications

• Strong written and communication skills.
• Publications, open-source work, or public write-ups are a plus.
• Deep expertise in interpretability, evals, AI safety, anomaly detection, or representation learning.
• Good judgment about which experiments are worth running and when a result is real.
• You ship experiments and build prototypes.
• You use and build AI agents and tools.

Responsibilities

• Set Raindrop’s research agenda and run the experiments behind it.
• Design evaluations and metrics for problems where the ground truth is unclear.
• Publish findings as papers, blog posts, and talks.
• Prototype new methods, and bring the ones that work into the product when it makes sense.
• Help define what research looks like at Raindrop as the team and product grows.

Skills

Interpretability

Evals

AI safety

Anomaly detection

Representation learning

Job description

About Raindrop

Raindrop is the monitoring platform for AI agents. Engineering teams at Fortune 100s and the fastest-growing AI companies (Vercel, Speak, Clay) use it to catch silent failures in production. When an agent misbehaves, Raindrop alerts the team, links to the trace, and surfaces the root cause so they can fix it fast. With Raindrop 2.0 (Self-Healing Agents), the loop closes itself: your coding agent fixes it, and the fix becomes an eval to prevent regressions.

Why It Matters

AI agents fail constantly in ways both hilarious and terrifying. Regular software throws exceptions. But AI agents fail silently, leaving engineers with almost no visibility into how their agents are actually performing.

The current status quo is sifting through millions of logs and trying to debug flaky evals that just aren't matching real world results. Evals are like unit tests, they confirm your model got specific test cases right. But in the real world agents call thousands of tools, run for hours, and encounter millions of unpredictable actions.

That’s where Raindrop comes in. It learns the unique shape of each AI agent’s issues. Starting from presets like Laziness, Forgetting, or Task Failure, to automatically tuning itself to each agent and discovering unknown unknowns.

With one click of a button, AI engineers start tracking issues or topics across 100% of their production data. They can see frequency over time, how many users are affected, relevant properties and more.

In order to process hundreds of millions of events, we gradually train small, custom models, private to each company, that learn to uniquely understand how their product is used.

As part of the early team, you’ll play a fundamental role in shaping the company - from making strategy and product decisions, to helping scale the team, to shaping the future of AI agents.

Our Investors

We've raised $50M total, led by CRV, with backing from Lightspeed, Figma Ventures, Vercel Ventures, and the founders of Replit (Amjad Masad, Michele Catasta), Cognition (Walden Yan), Framer (Koen Bok, Jorn van Dijk), Speak (Andrew Hsu), and Notion (Akshay Kothari).

About the role:

We’re looking for a founding Research Engineer to study how AI agents behave in production. You’ll join an early team, set the research agenda, and publish what we learn.

Your Focus:

• Agent behavior: Discover how agents fail and respond to different conditions in the real world, and what that means for monitoring and safety.
• Simulations: Advance how we simulate agents and the situations they face before they reach users.
• Detection: Develop new methods for finding issues no one has defined yet.
• Open questions: High-upside bets on questions about agents we haven’t thought to ask yet.

Your responsibilities:

• Set Raindrop’s research agenda and run the experiments behind it.
• Design evaluations and metrics for problems where the ground truth is unclear.
• Publish findings as papers, blog posts, and talks.
• Prototype new methods, and bring the ones that work into the product when it makes sense.
• Help define what research looks like at Raindrop as the team and product grows.

Ideal Candidate:

• You make complex ideas clear through strong written and communication skills.
• Care about sharing your research through publications, open-source work, or public write-ups.
• Deep expertise in an area such as interpretability, evals, AI safety, anomaly detection, or representation learning.
• Good judgment about which experiments are worth running and when a result is real.
• You ship experiments and build prototypes
• Use and build AI agents and tools