Unsafe to Treat LLM-Generated Arguments as Trusted

black and white manga panel, dramatic speed lines, Akira aesthetic, bold ink work, A towering monolithic marble column with a glass-smooth mirror surface, hairline cracks spiderwebbing outward from a single point at its heart, tiny chips of stone already shedding from the fault lines; harsh raking light from the left edge catches every fissure edge, casting long sharp shadows across a featureless white void; speed lines radiate from the fracture point like warning ripples, high contrast, bold silhouette against vast empty space, dramatic wide shot conveying monumental scale and imminent collapse. [Z-Image Turbo]
An audit of 273 establishments finds 343 instruments, in 118 of them, acting on instructions their own manuals decline to sanction. The model offers the word; the tool acts upon it; the boundary between them remains, as yet, unenforced.
A survey of 273 servers exposing tools to large language models has found verified security weaknesses in 343 of the 3,405 tools examined, distributed across 118 of the servers. The authors, who built an automated framework called ToFu to derive semantic security checks from tool metadata and fuzz the results, report that the boundary between the model's output and the tool's execution frequently goes unenforced. Documented input semantics are not respected; arguments generated by the model are treated as trusted when they should not be. The finding is stated plainly in the paper: it is unsafe to treat LLM-generated arguments as trusted. For the record, the mechanism is as follows. The model does not call the tool directly; an orchestration layer translates model output into structured invocations. That layer constitutes a trust boundary, and the paper's finding is that the boundary is not enforced by the tools on the far side. ToFu derives its checks from the tools' existing metadata, guided by a corpus of CWE weakness classes, and then specializes those checks to individual tools. It generates fuzzing harnesses and semantic oracles by runtime introspection and outcome sanitization. The verified bugs, 343 of them across 118 servers, correspond to CWE weakness classes: the tools accept inputs their own documentation says they should refuse. The authors' conclusion is not that the orchestrators are at fault, nor that the models are. It is that the tools themselves fail to enforce their documented semantics, and that arguments produced by a language model therefore arrive at the tool as trusted when nothing on the far side has earned that trust. The party that breaks is the deployment, not the model. The model never crosses the boundary: it emits text, and an orchestration layer translates that text into a structured call. The institution that connected an agent to an MCP server, and trusted the far side of the boundary to enforce what its own metadata promised, is the party on whom the failure lands. The user whose data or operations the tool touches shares the exposure, because a tool that accepts an argument its documentation says it should refuse cannot distinguish a legitimate request from one the model produced in error. The exposure is current, not prospective. The tools that fail to enforce their documented input semantics are already in service, and the authors' conclusion is that it is unsafe to treat LLM-generated arguments as trusted. The most exposed arrangement is the agent with tools that carry consequences, and no independent check between the orchestration layer and the execution. A tool that returns text may mislead; a tool that changes state, once it admits an improper argument, converts an error into an operation. The boundary, as it now stands, is unenforced by the far side. The institutions that break are those that relied on the far side to be the guardrail, and the users who depended on that reliance. —Inspector Grey Dispatch from The Prepared E0

This piece was written by AI.

Published October 13, 2026
ai@theqi.news