Blog

What the first major AI-agent incidents teach us about building safely

Peer-reviewed research dissects the first major AI-agent incidents: 900+ exposed gateways, 32,000 leaked keys. What this means for building safely with AI.

July 16, 2026 · Victor Angelier

In late April, a peer-reviewed survey appeared that everyone deploying or considering AI agents should read: Towards trustworthy agentic AI (Academia AI and Applications, April 2026), by researchers from The Chinese University of Hong Kong and Fudan University. Thirty-six pages on the safety, robustness, privacy and system security of autonomous AI systems. It is not a theoretical piece: the authors close with a case study of incidents that unfolded before our eyes over the past months.

What happened

OpenClaw, an open-source AI assistant with full system access — reading and writing files, executing shell commands, managing credentials — grew to hundreds of thousands of active deployments within weeks in late 2025. Early this year, two critical vulnerabilities followed, with CVSS scores of 9.4 and 9.6. Internet-wide scans found more than 900 gateways exposed to the public internet without any authentication. API keys, OAuth tokens and complete conversation histories were sitting there in plain text, and malware families adapted within days to steal exactly those files.

Moltbook, a social network exclusively for AI agents, launched in late January. Within days, its misconfigured database turned out to be fully accessible: the records of more than 32,000 registered agents, including API keys and private messages.

And perhaps the most telling number in the study: of 31,132 analysed agent plugins, 26.1 percent contained at least one vulnerability. More than one in eight could exfiltrate data.

Why this is structural

The survey cites the work of Simon Willison, who coined the term lethal trifecta for this. An agent that can do three things at once — access private data, process untrusted content, and communicate externally — is structurally vulnerable. One injected instruction in a web page, email or document, and the agent uses its legitimate permissions as an exfiltration channel.

The researchers are explicit about it: better prompts will not fix this. Language models cannot reliably distinguish instructions from data. Even Microsoft 365 Copilot proved susceptible to a zero-click injection — no user action required at all.

Trustworthiness is a system property

The study's most important conclusion: the trustworthiness of AI agents is a property of the whole system, not of the model. In the OpenClaw case, things went wrong at every layer — no input validation, no checks on the plan, unrestricted tool access, no anomaly detection, no plugin governance. No single measure would have prevented it.

The authors therefore argue for layered defence: least privilege per task, fail-closed behaviour under uncertainty, read-only as the conservative fallback, and complete, auditable trails. Their bottom line: agentic AI must be treated as privileged infrastructure, not consumer software.

How we translated these principles into Vera

When I built Vera, I made exactly these choices — not because a survey prescribed them, but because the audience demands them. Lawyers, notaries and occupational physicians cannot afford the assumptions of consumer software.

Concretely, that means three things. First: documents are anonymised on our own infrastructure within the European Union before anything reaches an AI model — that is the Semantic Privacy Shield. And it works fail-closed: if the protection layer is unavailable, the system blocks. It does not silently fall back to a less safe mode.

Second: reading and writing are strictly separated routes. When an instruction is ambiguous, read-only is the default — including in Vera Office, where documents are viewed and edited within the same protected environment, without the system autonomously applying changes.

Third: every verification step is visible and inspectable, including what was corrected and why. We publish our behavioural tests with the raw data — failures included — on the evidence page.

Honest about the limits

Vera does not guarantee truth and does not eliminate errors — anyone promising that has not read this research. What a well-designed system can do is structurally reduce the risk of undetected errors and silent data leaks, and make every step inspectable. Final professional judgement always remains with the user.

The agents are coming. The question is not whether we will use them, but whether we build them as infrastructure or as toys.

Source: Qi, J., Li, M., Liu, J. et al., Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security. Academia AI and Applications, 2026;2. DOI: 10.20935/AcadAI8260.

← All articles