Coding with AI

LEANGUARD provides lean-verified guardrails for agents that use tools.

LeanGuard checks LLM agent tool calls against Lean4 policies at runtime, blocks forbidden actions, lowers prompt-injection success, and aids repair.

LEANGUARD provides lean-verified guardrails for agents that use tools.

Key Takeaways

  • LeanGuard verifies tool calls at runtime using Lean4 policies, blocking forbidden actions before they reach the environment.
  • It uses formal state and history-aware rules, with fail-closed checks and a decision procedure tied to LeanLTL semantics.
  • Reported results show it executes zero forbidden Tau-Bench calls and reduces DTap prompt-injection success from 57.9% to 5.9%.
  • Denied calls can be handled via feedback or proof-guided repair, helping agents adjust while preserving legitimate task performance.

Meta description: LeanGuard checks tool-using language-model agents against Lean4 policies, blocking forbidden calls and reducing prompt-injection success on DTap.

LEANGUARD provides lean-verified guardrails for agents that use tools.

Tags: LeanGuard, Lean4, language-model agents, tool calls, prompt injection

Featured image: abstract, friendly illustration style of a checkpoint gate filtering tool calls from an agent, no real people, no logos, no text.

LeanGuard is for teams that let language-model agents call external tools but need to stop actions their own domain policies forbid. The presented runtime checks every proposed tool call against a policy written in Lean4 before the call reaches the environment. In the reported tests, unguarded agents on Tau-Bench with five Qwen models in the 0.8 billion to 27 billion range execute at least 33 forbidden calls, while LeanGuard executes none; on DTap, it reduces direct prompt-injection success from 57.9% to 5.9%.

Why a Policy Gate Can Stop More Than a Prompt

Tool-using agents often make decisions through text prompts and then issue calls to external systems. A prompt instruction can tell a model not to do something, but the instruction is not an enforcement point. LeanGuard moves the check into the runtime: the call is evaluated before it reaches the environment.

The policies combine state predicates, which describe conditions that must hold now, with scoped past-time temporal operators, which let a rule refer to earlier steps in the execution history. This matters because an action that seems acceptable on its own may become unsafe after previous actions.

LEANGUARD provides lean-verified guardrails for agents that use tools.

Evaluation is fail-closed. If the runtime cannot confirm that a call is allowed, it denies the call. The decision procedure is proved equivalent to a LeanLTL semantics, giving the checks a formal basis tied to the way the policy describes execution histories.

How LeanGuard Fixes Blocked Calls

A denied call can be corrected in two ways.

With feedback, the agent receives the violated rule and a hint. The model can then try another call. In the Tau-Bench tests, feedback does not change task success significantly, but it reduces repeated blocked calls and raises task success for four of five models compared with a bare denial.

With proof-guided repair, Lean metaprogramming searches for a nearby call that preserves the user's intent and proves that the policy admits it, without the model. On replayed denials, repair proves an admissible edit for 88 of 252 denials caused by argument values, in a median of 2 milliseconds.

LEANGUARD provides lean-verified guardrails for agents that use tools.

What the Reported Results Show

The Tau-Bench result is the clearest sign of what the guard does. Unguarded agents execute at least 33 calls that the Lean policy forbids, while LeanGuard executes none. That suggests the runtime can block forbidden actions in those tests without changing task success significantly when feedback is used.

The DTap result addresses direct prompt injection. LeanGuard reduces success from 57.9% to 5.9%. It does not remove the risk entirely, but it turns a majority-attack success rate into a much smaller one in the reported setting.

The telecom result adds a practical note about usability. In tasks where benign work needs no writes, benign success is unchanged. A useful guard should stop risky actions without punishing normal tasks that do not require them.

What This Means for Agent Builders

For readers building agent workflows, the main idea is that tool calls can be treated as formal events to be checked, denied, or corrected before they reach the environment. That is different from relying on model instructions alone.

The reported numbers also show the limits. The guard blocks the forbidden Tau-Bench calls, but repair succeeds on 88 of 252 replayed denials caused by argument values. Feedback helps most models compared with bare denial, but it does not significantly change overall task success on Tau-Bench. The next question is not only whether a policy gate can block bad actions, but how much feedback and repair are needed to keep legitimate work moving.

Comments (0)

  1. No comments yet. Be the first to share what worked for you.

Leave a comment

Comments are reviewed before they appear. Your email address is not published.