AI agents use evidence-guided recovery when tool outcomes are ambiguous and execution is partial
Evidence-guided recovery helps AI agents resume safely after ambiguous tool timeouts by using evidence to separate safe stopping from goal completion.
Key Points
- Tool timeouts create ambiguous recovery because the agent cannot tell whether the action failed, committed, or remains in flight.
- The reported method is evidence-guided recovery: agents resume only within approved obligations, declared provider capabilities, and states supported by public evidence.
- Safety is enforced through admissibility checks for expiry, authorization, dependencies, and residual effects, plus a terminal certificate that separates safe completion from full goal satisfaction.
- In the evaluated studies, planning advice outperformed evidence prompting: 30 certified completions versus 9 across 58 paired units, with a 36.2-percentage-point difference and p=0.0078.
- The method is not a universal fix for timeouts; its guarantees depend on declared provider capabilities and explicit handling of unknown or handoff outcomes.
AI agent tool timeouts are the problem addressed by a reported evidence-guided recovery method for partial execution. The method matters to readers who evaluate how agents use external tools, because a timeout can leave an action failed, committed, or still in flight while later workflow steps have already completed. The source gives concrete counts: in 37 assignments, planning and three active controls each certify 28 completions. In a later comparison across 58 paired units in 19 previously developed task families, planning advice certifies 30 completions versus 9 for evidence prompting, a 36.2-percentage-point difference with exact family sign-flip p=0.0078.
Why ambiguous tool outcomes are hard to recover
The core difficulty is not that a tool call takes too long. It is that the agent cannot know from the timeout alone whether the tool did nothing, did something that cannot be undone, or is still finishing. If the workflow has already moved forward, the agent may be planning against a world that does not match its last observation.
The reported approach is framed as evidence-guided task recovery over approved obligations, provider capabilities, and states compatible with public evidence. In plain terms, the agent is asked to resume only within what the available evidence supports, not to assume that an action did or did not succeed beyond what can be justified.
This framing matters because recovery can become unsafe if the agent guesses. A guessed retry may duplicate an action. A guessed abort may leave partial effects unhandled. The source therefore builds the method around checks that can keep such guesses from becoming hidden assumptions.
What the recovery check keeps separate
A shared admissibility check handles key expiry, authorization timing, dependencies and residual effects. These are practical failure modes in tool-using systems: a credential may expire, an authorization may arrive too late, later steps may depend on earlier ones, and an action may leave effects behind even if it does not appear to complete.
The terminal certificate separates actual goal satisfaction from publicly supported safe completion. That distinction is one of the most useful parts of the report. An agent can be certified as having reached a publicly supported safe stopping point without being claimed to have fully satisfied the original goal. The report keeps those outcomes explicit rather than collapsing them into a single success label.
The known-model implementation combines standard observation-contingent planning with a separately implemented effect-history adjudicator. Observation-contingent planning means the plan adapts after new observations. The effect-history adjudicator is described as separate, which suggests the system does not rely on the planner alone to decide what effects already happened.
What the reported studies found
The first result is a finite study of 37 assignments. Planning and three active controls each certify 28 completions. The report also notes that four saved synthetic fee units incur additional computation. The source does not define what those units represent, so the safe reading is that they are a reported accounting detail tied to computation, not a benchmark score to interpret further.
The second result is a preregistered assignment-held-out comparison. It evaluates planning advice against evidence prompting with two local model bundles. Across 58 paired units in 19 previously developed task families, planning advice certifies 30 completions versus 9 for evidence prompting. That is the 36.2-percentage-point difference reported with exact family sign-flip p=0.0078.
The report also gives resource and verification numbers: planning advice uses 272 proposals versus 399, and all 116 trajectories pass independent replay. Both models exhibit a positive completion difference. Independent replay is important because it means the reported trajectories were checked separately from the original run.
What the evidence still does not settle
The comparison measures the evaluated advice interface under a common shield. The source states that search-versus-rule advice requires a separate matched comparison. That caveat keeps the result from becoming a general claim that all planning-based advice beats all evidence-based prompting in every agent setting.
The guarantees are also tied to declared provider capabilities rather than arbitrary third-party exactly-once execution. This is a key boundary. The method does not claim that any external tool can be forced to behave as if every action executed exactly once. It works within the capabilities that the provider declares, and it leaves unknown and handoff outcomes explicit.
For readers, the takeaway is not that AI agents now have a universal fix for tool timeouts. The reported value is more specific: a recovery design that tries to keep partial execution from being read as certainty, that separates safe completion from goal satisfaction, and that shows measurable improvements in the evaluated setup.

Comments (0)