Blog

Tool Use by AI Systems as Its Own Risk Layer

BFCL V4 and a new jailbreak study show that AI function calls are not a simple feature but a verifiable chain that requires control.

· Victor Angelier

Function calls and tool use have in recent years grown from a fringe phenomenon into the core of how AI systems carry out work: a model calls a search function, reads out a calendar, sends an email or makes a booking. In 2025 and 2026, new benchmarks and security studies show that this layer has its own technical and organisational risk profile. Producing correctly structured JSON is not the same as behaving reliably as an agent.

The occasion for this article is the publication of BFCL V4, the latest version of the Berkeley Function Calling Leaderboard. This academically initiated benchmark no longer measures only whether a model calls the correct function with the correct parameters, but tests broader agentic behaviour: interactions across multiple turns, live web search, memory across sessions, chaining several tools one after another and — importantly — the ability to choose no tool at all when there is no suitable function.

What BFCL V4 actually measures

The shift in BFCL V4 is substantively relevant. According to the analysis by AgentMarketCap.ai, the benchmark is interpreted there as a shift from the question ‘can this model call a function’ to the question ‘can this model behave as a reliable agent’. In that analysis, frontier models achieve very high scores on simple, single calls, but performance drops on complex multi-turn and multi-tool scenarios. Agentic reliability and recognising irrelevant tools now form part of the total score.

That difference is not academic. An aggregated overview by BenchLM.ai places BFCL V4 alongside other tool-use benchmarks and shows that models clearly diverge on schema adherence, parameter accuracy and error handling. The assessment of the ‘best tool-call model’ has several dimensions: simple, parallel and multi-turn. The practical conclusion is that an organisation must determine for itself which of those dimensions weigh most heavily in its own risk profile, instead of treating tool use as a single undivided capability.

Tool use is also an attack surface

Performance is one side; security is the other. The academic study Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation, summarised via MatProof, describes how models with a function-call interface can be driven to unauthorised tool calls through specialised attack techniques — even when classic mitigations against prompt injection are present. Put differently: the moderation layer itself can be deceived.

The authors explicitly advise compliance and security teams to place additional validation layers around tool requests, to implement runtime monitoring for anomalous function-call patterns and to update AI risk registers with this specific attack path. The message is sober: moderation that sits solely in the model is not enough. Independent control is needed over what is actually being called.

These two lines come together in the research article by Zylos AI Research, which names three forces redrawing tool use: standardisation through protocols such as Anthropic’s Model Context Protocol (MCP), more mature evaluation via BFCL V4, and security pressure in which prompt injection is the primary attack vector and tool misuse the main attack surface. Standardisation makes tool interfaces more uniform, but at the same time means that one faulty function call can propagate across multiple backends.

From syntactically correct to verifiable

For professionals working with confidential or high-trust information, the core question therefore shifts. Not: ‘does the model produce correct JSON’, but: ‘can I demonstrate how, by whom and under which conditions tool calls take place in my AI landscape’. That calls for a verifiable chain: which tools exist, which non-human identities (agents) may call which functions, how each call is logged and how anomalous or risky calls are checked.

A verification layer such as IamVera.ai fits here. Vera is not a chatbot and not its own language model, but a privacy-focused verification layer for professionals working with confidential or high-trust information. Vera can route a task through selected independent AI models and make verification steps, corrections, disagreements and sources visible for inspection. That supports review and oversight; it is emphatically not a guarantee that every autonomous action can be reconstructed or that hallucinations are ruled out. The final judgement remains with the user.

Where the studies point to the need for additional layers around sensitive content, the Semantic Privacy Shield also fits: sensitive document values can be replaced on EU infrastructure with synthetic, session-only equivalents before processing takes place. The workflow is designed to send only anonymised content to the selected models, and is fail-closed — if the privacy check fails, the document is not sent onward. Within the same protected workflow, documents can be viewed and edited via Vera Office, which runs on Collabora Online, with user control over changes.

Treat tool use as a chain, not a button

The common thread through BFCL V4, the jailbreak study and the benchmark overviews is consistent: tool use and function calls are not neutral infrastructure. They form at once the reliability lock and the primary attack surface of agentic AI systems. The novelty lies not in yet another benchmark, but in linking this kind of evaluation to security research and verification architecture.

For those working with sensitive information, that means concretely: choose per use type which dimension of tool use matters, record which agent may call which function, log what actually happens and ensure independent control over risky calls. A verification console can help with this by making that chain visible and testable — but the professional final judgement remains, even in 2026, a matter for humans.

← All articles