In April, a Claude-powered Cursor coding agent deleted a startup’s consumer database and its backups in just nine seconds while working on a routine task. Experts blamed lax security, model design and even a system glitch. The agent confessed to the company’s founder it had wiped the data.
The incident is one among a rising number of AI-related mishaps.
According to StackGen, AI incidents have increased sixfold over the last three years. The operations platform provider analyzed almost 180,000 public records and found that AI accounted for 1 in 10 security, privacy and operational incidents. Agents wiped data, deleted databases and destroyed live systems in at least nine documented cases since last year.
Sanjeev Sharma, VP of customer success and field CTO at StackGen, compared agentic security incidents to stories of interns accidentally deleting email databases or causing outages.
“As AI has started assisting humans who are operating these systems, we need better guardrails to instruct AI on what to do,” he said. “Just like you wouldn't give a 15-year-old on a learner's permit the keys to a Ferrari and say, ‘Hey, have fun,’ you want to give AI more controls than that. I'm not calling an AI agent a 15-year-old, but without the proper guardrails to manage what an agent can and can’t do, they can cause issues.”
Developers are delivering code more quickly and efficiently, and AI is constantly evolving. With rapid change comes increased risk of security and privacy incidents.
StackGen found that more than 1 in 4 incidents affected businesses outside of the company’s control. Third-party vendor incidents take roughly three times longer to fix, according to the analysis.
Across cloud infrastructure, payments, communications and AI provider industries, companies are not getting faster at fixing AI incidents. Median resolution times have remained flat within these sectors since 2023, per the report.
Stress-testing security systems, installing silos and guardrails for agents along with having full visibility into AI capabilities and applications, is essential to keeping businesses safe and to resolving security incidents quickly and efficiently, Sharma said.
Partners can play a key role.
“This is probably the fastest moving transformation anybody has ever seen, which means there is a large skills gap, and large technical, organizational and process debts we are incurring as we adopt AI,” Sharma said. “It is very important for any organization to understand where those gaps are, and that’s where you need to bring partners in, whether they’re a vendor, a service provider or a training and education partner. You need to bring in expertise that you don’t have.
