Raindrop Secures $50M in Funding to Address AI Failures in Production
Raindrop has raised a Series A led by CRV, securing a total of $50 million in funding, though the round size remains private. The San Francisco-based company focuses on monitoring and catching silent failures in live AI agents. It launched Simulations, a tool that tests changes against real traffic before deployment.
Addressing a Growing Problem
"Agents now run for hours, call thousands of tools, and handle real money, real health data, and real customers," stated Zubin Koticha, Raindrop's CEO. "When an agent fails, it does the wrong thing convincingly at scale until someone happens to notice."
The company analyzes production traces to identify what it calls semantic anomalies—hallucinated answers, tool misuse, and behavior changes that arise with model upgrades. Engineering teams can then pinpoint what changed, when it started, and which users were affected.
Introducing Simulations
Alongside the funding announcement, Raindrop introduced Simulations, a research preview product. It replays real production traffic and existing test cases against proposed agent changes, running anomaly detection on the results. This approach aims to catch behavior changes that conventional evaluations might miss since they rely on test cases written in advance.
Raindrop draws parallels to testing processes used internally by frontier labs like OpenAI and Anthropic. It cites METR research showing how rapidly AI agents are expanding their capabilities, doubling their task completion time roughly every seven months.
Investor Support and Customers
Lightspeed Venture Partners and Y Combinator joined existing investors in backing the round, along with lead researchers from OpenAI, Anthropic, and Thinking Machines. Figma Ventures and Vercel Ventures are also listed as backers.
Raindrop counts Vercel, Framer, and Clay among its customers, as well as unnamed Fortune 100 enterprises in healthcare and logistics. Reid Christian of CRV described agents as "fundamentally different from traditional software," stating that Raindrop treats agent failure as a detection problem akin to security issues.
Bucky Moore of Lightspeed echoed this sentiment, emphasizing the high-stakes nature of AI failures in sectors like defense. "If we’re having an issue like a build failure or agents stuck in a loop, we see that issue in Slack," added Bani Singh, an AI engineer at Vercel.
About Raindrop
Founded by Zubin Koticha, Ben Hylak, and Alexis Gauba, Raindrop's team includes engineers from Robinhood and Square who have built fraud detection models and anomaly detection systems. Security engineers from Segment, Semgrep, and Socket.dev have also joined the company.