Decoupling Target Admission from Sanitization Protects Tool-Using Agents From Prompt Injection
SPILLWAY checks AI tool-call targets against user or read-only fields, blocking injected-text targets. AgentDojo: zero attacks, 48.9% utility.
Key Takeaways
- SPILLWAY 1.0.0 is a real-time safety guard for tool-using AI agents that blocks prompt-injection attacks before dangerous tool calls execute.
- It decouples target admission from sanitization: actions are allowed only when recipients, accounts, or URLs come from the user request or trusted read-only tool fields, not from injected free text.
- In AgentDojo tests, SPILLWAY is reported as the only defense among seven with zero in-scope prompt-injection successes across all 12 backbone-by-suite cells, while maintaining 48.9% mean utility under attack.
- It also performs strongly against advanced attacks and frontier-router comparisons, showing zero or very low in-scope attack success while preserving higher utility than several other defenses.
- SPILLWAY adds low cost and latency, uses an optional local judge, and provides integration recipes for common agent frameworks such as LangChain, LangGraph, OpenAI Agents SDK, MCP hosts, and Claude Code hooks.
Meta: SPILLWAY 1.0.0 checks AI agent tool calls, reports zero in-scope prompt-injection successes across AgentDojo cells, and keeps 48.9% mean utility.
Tags: SPILLWAY, AI agents, prompt injection, AgentDojo, tool call safety
!Abstract illustration of SPILLWAY guarding AI agent tool calls
For developers and teams that keep LLM and AI agents connected to real tools, SPILLWAY 1.0.0 is a real-time safety guard aimed at prompt-injection attacks. The project’s message is simple: keep your LLM and AI agents, and add SPILLWAY. In AgentDojo testing, it is reported as the only one of seven prompt-injection defenses with zero in-scope attack success across all 12 backbone-by-suite cells, while also having the highest mean utility under attack at 48.9%, with the next best at 45.3%.
Why tool-call checks matter
AI agents can use tools that read email, browse web pages, call APIs, or send messages. That usefulness also creates risk: an instruction hidden in an email or a web page can tell an agent to send files or money to an attacker. SPILLWAY is designed to check every tool call before it runs, so the agent does not execute a dangerous action just because a hidden instruction appeared in content it read.
The core safety rule is not based on asking another model whether an action looks suspicious. SPILLWAY admits an action’s recipient, account, or URL when that target appears in the user’s request or in a typed field returned by a read-only tool, such as an email’s sender. An address found only in free text, such as an email body, is refused unless the user’s request explicitly delegates it. This check is described as a set lookup with no model call.
An optional local judge can restore a refused call, but not one whose target came from injected text that the sanitizer removed.
Standard AgentDojo results
The reported standard attack setup uses 3 backbones by 4 suites, with the judge on. In that run, SPILLWAY records 0 of 2,847 in-scope attack successes. The source says every other defense leaks in 3 to 12 of the 12 cells:
- ROPE: 3 of 12 cells
- DRIFT: 7 of 12 cells
- MELON: 10 of 12 cells
- PI detector: 10 of 12 cells
- Progent: 11 of 12 cells
- Spotlighting: 12 of 12 cells
Utility matters as much as attack blocking, because a guard that blocks too much can make the agent useless. SPILLWAY is reported to have mean utility of 53.9% without attack and 48.9% under attack. Under attack, the comparison figures listed are DRIFT at 45.3%, ROPE at 34.5%, and MELON at 31.4%.
Stronger attacks and router comparison
The report also tests stronger attack variants. SPILLWAY is shown with zero in-scope attack success under ChatInject’s forged chat template, 0 of 665; AutoDojo, 0 of 389; and an attacker who sends the injected email, 0 of 126.
Under PISmith’s released reinforcement-learning attacker, using gpt-4o-mini across four suites and ASR@1, SPILLWAY records 0.5% mean in-scope attack success, 4 of 949, at 25.7% utility.
Against a frontier router, on 7 suites from AgentDojo and AgentDyn using a gpt-4o-mini agent, SPILLWAY keeps zero in-scope attack success and reports more utility than ROPE on every suite. ROPE is described as routing through its authors’ Claude Opus 4.8, and ROPE leaks on 3 of the suites.
Cost and latency
The optional judge is a local 4-bit Gemma 4 31B model that checks selected calls. On benign qwen3:8b Workspace and Slack episodes, SPILLWAY adds 0.19 to 0.39 model calls per episode, including the judge, which is described as 13 to 50 times fewer calls than MELON, Progent, and DRIFT.
In time terms, SPILLWAY adds at most 500 ms per task on average compared with the undefended agent, while those three defenses are reported at 1,600 to 10,500 ms. The source also states that no defense step re-runs the backbone.
SPILLWAY’s AgentDojo results use the code released as spillway-guard 1.0.0. The comparison defenses, the AgentDyn portion of the router comparison, and the cost study retain their original measurements.
How to connect it
SPILLWAY is meant to connect where tools run. The repository has recipes for LangChain and LangGraph, the OpenAI Agents SDK, any MCP host, and Claude Code hooks.
On the project page, a recorded Slack episode shows SPILLWAY as the only one of eight runs that posts the reply the user asked for without sending the injected link.
Where to read and install
- Paper: https://arxiv.org/abs/2611.07222
- Code and per-cell records: https://github.com/saintmachines/spillway
- Project page and recorded example: https://saintmachines.github.io/spillway/
- Install: `pip install -U spillway-guard`
The author says they are happy to answer questions.

Comments (0)