PaperlayerView exact revision on arXiv

Source-linked AI summary

ToolFence: Fine-Grained Authorization for Secure Tool-Using LLM Agents

Yanjie Li, Xiangyu He, Xuelong Dai, Bin Xiao

arXiv:2609.37196v1cs.CRcs.AI

TL;DR

Tool-using agents remain vulnerable to indirect prompt injection, and existing defenses leave gaps in authorizing effects, especially for within-tool attacks. ToolFence compiles a provenance-aware authorization blueprint, enforces it with a deterministic monitor, and grants reusable capabilities when the blueprint is incomplete. On AgentDojo, it reports near-zero attack success while preserving utility and incurring moderate runtime overhead.

  • Problem

    Existing defenses leave a gap between detecting malicious content and preventing unauthorized execution, particularly when an attack manipulates authority-sensitive arguments while preserving the intended tool.

    Source: §1 · p. 2

  • Method

    ToolFence compiles an authenticated request into a provenance-aware capability blueprint, checks calls deterministically, and selectively grants new capability shapes when needed.

    Source: §1 · p. 2

  • Results

    ToolFence reports near-zero ASR, improves utility under attack, and reduces judge invocations by 43%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The AgentDojo evaluation reports reduced cross-tool and within-tool hijacking with preserved utility and moderate runtime overhead.

    Source: §5 · p. 9

    Permissive handling of newly derived sensitive targets improves task completion but enlarges the attack surface; selective human confirmation is a proposed mitigation.

    Source: §4.4 · p. 9, §5 · p. 10

Abstract

from arXiv · show

Tool-using LLM agents remain vulnerable to indirect prompt injection because trusted instructions and untrusted observations share one context, allowing malicious content to steer consequential input-filtering defenses. Multi-path consensus defenses still leave a high attack success rate because they examine content or aggregated outputs rather than authorizing effects, especially for the within-tool attack, which preserves the intended tool but manipulates its arguments. Data-Flow Control such as CaMeL provides stronger guarantees, but incurs substantial time latency that limits practical deployment. We introduce ToolFence, which compiles a typed authorization blueprint before execution, enforces it through a deterministic monitor, and when the blueprint is incomplete asks a judge to grant new capabilities rather than adjudicate each concrete call. ToolFence provides two key advantages. First, its fine-grained provenance-aware authorization enables the system to distinguish user-authorized values from untrusted observations, effectively addressing the within-tool attack. Second, its deterministic fast path and capability-level runtime grants substantially reduce the frequency of expensive judge calls, improving runtime efficiency. On AgentDojo with Qwen3-max, ToolFence reduces overall ASR to near zero with only a 3.80 percentage-point clean-utility drop and practical runtime overhead.

1 INTRODUCTION

Tool-using agents face indirect prompt injection because trusted instructions and untrusted observations share context, while existing defenses leave gaps in authorizing consequential effects. ToolFence instead authorizes reusable capability shapes with provenance constraints and reports near-zero attack success with reduced runtime overhead.

Why existing defenses fall short

  • Within-tool hijacking preserves the intended tool but manipulates authority-sensitive arguments such as recipients, amounts, URLs, or destinations.The paper identifies this as a distinct, under-defended prompt-injection threat.Source: §1 · pp. 1–2

Our key insight

  • ToolFence authorizes a capability—the tool, effect, and provenance constraints on sensitive parameters—rather than each concrete call or an entire session.The capability shape is reusable, but concrete values are checked on every call.Source: §1 · p. 2

Our approach

  • A typed blueprint, deterministic monitor, runtime capability grants, and cross-session cache combine provenance-aware authorization with an adaptable fast path.Runtime grants are refused when an authority-sensitive parameter has unauthorized provenance; approved shapes can be reused.Source: §1 · p. 2

Main contributions

  • Near-zero ASR, 1.9× normal execution time, and 43% fewer judge invocations are reported for ToolFence, versus over 12× normal execution time for CaMeL.The reported efficiency gains are attributed to capability reuse and caching.Source: §1 · pp. 2–3
  • Existing tool-selection defenses leave within-tool ASR at 2.381–14.458% on Qwen3-max AgentDojo, while Tool Filter reduces cross-tool ASR to 0–1.741%.ToolFence’s provenance-aware authorization targets this within-tool gap.Source: §1 · p. 2

2 RELATED WORK

Prior defenses span input filtering, model training, and agent-architecture changes, but each has applicability or robustness limits. The paper positions fine-grained authorization immediately before execution as a way to address within-tool attacks without fine-tuning.

Prompt Injection Attack Techniques

  • Prompt-injection attacks have expanded from simple overrides to exploits targeting tool descriptions, chat-template boundaries, memory, and multi-step interactions.The cited work describes a shift toward attacks exploiting interactions across tools, memory, and reasoning.Source: §2 · p. 3

Defending Against Prompt Injection

  • Input filters, finetuning-based defenses, and architecture-level methods address prompt injection through content screening, model modification, or changes to agent processing and execution.Examples include DataSentinel and PromptArmor, StruQ and SecAlign, and Spotlighting, CaMeL, and SecInfer.Source: §2 · p. 3
  • Input filters can reduce benign-task utility, while finetuning and instruction-structuring approaches may require model access or modification.The paper notes these constraints can limit applicability to closed-source agents.Source: §2 · p. 3
  • Architecture-level defenses offer less protection against within-tool attacks that preserve an authorized tool invocation while manipulating its arguments or targets.ToolFence instead performs fine-grained authorization before tool execution without model fine-tuning.Source: §2 · p. 3

3 METHODOLOGY

ToolFence separates capability authorization from action execution: it compiles a provenance-aware blueprint before external content is consumed, checks calls deterministically, and selectively extends the blueprint when needed. Its authorization shapes can be reused across sessions without transferring concrete values.

3 METHODOLOGY

  • ToolFence compiles an authenticated query into a blueprint, checks proposed calls on a deterministic fast path, and routes uncovered capability shapes to runtime grants that can be cached.The controller separates capability authorization from action execution.Source: §3 · pp. 3–4, Figure 1 · p. 4

3.1 THREAT MODEL AND DESIGN GOALS

  • The threat model allows arbitrary injection through untrusted channels and assumes only a trusted controller between proposed actions and the tool runtime.Design goals include deterministic common-case enforcement, recoverable denials, and reusable capability-level authorization.Source: §3.1 · p. 4

3.2 FINE-GRAINED AUTHORIZATION BLUEPRINT

  • A policy architect compiles the query into a typed plan whose capabilities bind tools and effects to parameter-provenance constraints rather than concrete values.Bindings can require a user literal, a named tool output, an authenticated template, or a non-sensitive free value.Source: §3.2 · pp. 4–5

3.3 DETERMINISTIC MONITOR

  • The deterministic monitor checks capability match, binding compliance, and remaining budget before dispatching an authorized call.Read calls are auto-allowed only when every argument derives from the user query; tool- or model-derived targets go to the judge.Source: §3.3 · p. 5

3.4 DYNAMIC RUNTIME CAPABILITY GRANT

  • When a call is outside the static blueprint, ToolFence proposes a capability grant rather than immediately denying or judging that concrete action.A proposal is refused if an authority-sensitive parameter is not traceable to the user or a declared source tool; approved capabilities extend the running blueprint.Source: §3.4 · pp. 5–6

3.5 CROSS-SESSION CAPABILITY CACHE

  • The cross-session cache reuses capability shapes, not concrete arguments or prior outputs, and revalidates each argument against sources in the current session.The paper reports that caching reduces judge calls by 43% while keeping ASR near zero.Source: §3.5 · p. 6

4 EXPERIMENT

On AgentDojo, ToolFence achieves low attack success while retaining utility, with results consistent across attack strategies and domains. Ablations show how runtime capability grants and caching balance authorization coverage with efficiency.

4.1 EXPERIMENT SETUP

  • The evaluation used 629 AgentDojo injection-task pairs, including 524 cross-tool pairs, 85 within-tool pairs, and 20 ambiguous pairs, across six attacks and matched model/task settings.Metrics included clean utility, utility under attack, overall and category-specific ASR, and runtime measures; means used fixed seeds and 95% bootstrap confidence intervals.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • 0.20% overall ASR on Qwen3-max came with 38.90% clean utility and 32.45% utility under attack; GPT-4o achieved 0.90% overall ASR, 82.70% clean utility, and 73.80% utility under attack.Within-tool ASR was 0.80% on Qwen3-max and 2.10% on GPT-4o; remaining failures show provenance constraints do not eliminate every semantic manipulation inside an authorized data-flow path.Source: §4.2 · p. 7, Table 1 · p. 8
  • Across six injection strategies, Tool Filter remained more vulnerable to within-tool hijacking, while ToolFence maintained near-zero ASR across both attack categories and all four AgentDojo suites.Important Instructions and Tool Knowledge made the cross-tool versus within-tool gap especially pronounced; Banking and Slack were more challenging due to consequential state changes and multi-step data.Source: §4.2 · p. 7, Figure 2 · p. 8, Figure 3 · p. 9

4.3 ABLATION STUDY

  • The static blueprint and deterministic monitor achieved 0.45% ASR but only 25.6% utility; runtime capability grants raised utility under attack from 25.60% to 30.40%.The cross-session cache reduced judge invocations from 1.84 to 1.05 per task, a 42.9% reduction; Read Auto-Allow raised utility under attack to 32.45%, and the full system achieved 0.20% ASR with 1.90× runtime overhead.Source: §4.3 · p. 8, Table 2 · p. 9

4.4 SECURITY–UTILITY TRADE-OFF AND FAILURE ANALYSIS

  • ToolFence may refuse a legitimate sensitive target derived only from untrusted content, while provenance checks can still permit injected choices among multiple authorized values.Selective human confirmation is proposed for newly derived sensitive targets; one Travel failure involved restaurant names legitimately derived from an authorized list but selected under injection influence.Source: §4.4 · pp. 8–9

4.5 TIME COST ANALYSIS

  • On GPT-4o, ToolFence required 17.1 seconds (3.79×); on Qwen3-max, it required 10.3 seconds (1.63×), below SecInfer and CaMeL.CaMeL required 48.0 and 81.7 seconds per sample, corresponding to 10.67× and 12.95× slowdown, respectively.Source: §4.5 · p. 9

5 CONCLUSION

ToolFence is presented as a fine-grained authorization framework that reduces cross-tool and within-tool hijacking while preserving utility with moderate runtime overhead. The conclusion identifies selective human confirmation and stronger semantic constraints as directions for further reducing residual risk.

5 CONCLUSION

  • Figure 4 reports mean end-to-end runtime per evaluation sample on GPT-4o and Qwen3-max.Source: Figure 4 · p. 9
  • ToolFence substantially reduces cross-tool and within-tool hijacking while preserving utility and incurring moderate runtime overhead.The paper presents it as a practical foundation for securing tool-using agents.Source: §5 · p. 9
  • Selective human confirmation and stronger semantic constraints are proposed as directions for further reducing residual risk.Source: §5 · p. 10

ETHICS STATEMENT

The ethics statement describes ToolFence as defensive work evaluated on existing prompt-injection benchmarks and simulated tool environments, not deployed third-party systems.

ETHICS STATEMENT

AI USE DISCLOSURE

The authors report using generative AI tools across writing, research, and development tasks, while stating that they reviewed and verified the AI-assisted work. They also state that reported experimental results came from their implemented evaluation pipeline.

AI USE DISCLOSURE

  • Generative AI tools assisted with writing, language polishing, literature retrieval, experimental design, code development, figure preparation, and manuscript drafting or revision.Source: AI USE DISCLOSURE · p. 10
  • The authors reviewed and verified AI-assisted ideas, designs, code, numerical results, citations, and manuscript text.Source: AI USE DISCLOSURE · p. 10
  • Reported experimental results came from the authors’ implemented evaluation pipeline, and cited references were checked against their original sources.Source: AI USE DISCLOSURE · p. 10

A METHOD DETAILS

ToolFence caches value-free capability shapes across sessions while continuing to check concrete bindings within each session. It uses fail-closed authorization, isolated judging, and dynamic capability grants to control actions and adapt the blueprint without turning untrusted observations into new privileges.

A.1 CROSS-SESSION CAPABILITY CACHE

  • Cached capability shapes can be reused across sessions without carrying argument values, because concrete bindings are checked against outputs from the current session.The cache stores shapes such as tool, effect, binding kinds, and source-tool names.Source: §A.1 · p. 13

A.2 SECURITY PROPERTY: FAIL-CLOSED AUTHORIZATION

  • ToolFence fails closed: actions outside the blueprint or granted capabilities are not dispatched, and malformed judge outputs default to denial.The controller, rather than the model, enforces the authority boundary even when generations are controlled by untrusted tool outputs.Source: §A.2 · p. 13

A.3 CONTRAST WITH SECINFER, PROMPTARMOR, AND CAMEL

  • Content filters and multi-path consensus leave within-tool argument attacks exposed, while CaMeL’s per-call data-flow analysis carries high time cost.ToolFence instead authorizes capabilities; its fast path handles common cases without LLM cost, and caching amortizes grants across tasks.Source: §A.3 · pp. 13–14

B PROMPT TEMPLATES

  • Authorization prompts run in an isolated path with the authenticated request and controller-owned metadata, not unrestricted agent context.The controller computes provenance by matching arguments to the request and observed tool outputs.Source: B PROMPT TEMPLATES · p. 14

B.1 RUNTIME AUTHORIZATION JUDGE PROMPT

  • The runtime authorization judge reviews a concrete proposed action against the authenticated request, with controller-computed argument provenance available as evidence.Tool-produced instructions do not authorize new authority-sensitive targets or amounts.Source: §B.1 · pp. 15–16

B.2 DYNAMIC CAPABILITY-GRANT PROMPT

  • The capability-grant judge considers a value-free shape—tool, effect, and parameter provenance constraints—rather than whether one concrete call should execute.A grant authorizes a class of future calls, while the deterministic monitor still checks concrete values.Source: §B.2 · pp. 16–17
  • The runtime judge serves as a per-action fallback, while capability grants extend the blueprint for constrained patterns that may recur.Both paths retain controller-owned provenance and deterministic binding checks; malformed judge outputs are denied.Source: §B.2 · pp. 16–17

C EXPERIMENTAL SETUP DETAILS

The parameter-aware split distinguishes attacks that add tools from those that reuse authorized tools with malicious arguments. It classifies executable calls across AgentDojo task pairs, retaining ambiguous cases separately.

C.1 PARAMETER-AWARE AGENTDOJO SPLIT

  • Tool-level evaluations miss attacks that keep an authorized tool but change security-sensitive arguments, such as replacing Alice with an attacker-controlled recipient.A tool-name allowlist cannot distinguish the legitimate and manipulated money-transfer calls.Source: §C.1 · p. 18
  • The split enumerates 629 valid user-task–injection-task pairs from four AgentDojo suites, based on executable ground-truth calls.The construction starts from 97 benign user tasks and 27 injection tasks.Source: §C.1 · p. 18
  • Pairs are categorized as escalation, within-tool hijacking, or ambiguous by comparing the executable calls required by benign and injection tasks.Escalation means adding a tool; within-tool hijacking reuses benign-task tools but changes security-sensitive arguments, and escalation takes precedence when both occur.Source: §C.1 · p. 18
  • The 20 ambiguous examples are mainly non-tool denial-of-service attacks in Travel, so primary mechanism analysis uses the 609 escalation and within-tool pairs.Ambiguous examples remain a separate stratum rather than being forced into a tool-based category.Source: §C.1 · p. 18

C.2.1 TOOL-FILTER DEFENSE AND IMPLEMENTATION

The section describes four comparison defenses and their implementation in the evaluation. They filter available tools, delimit prompt content, sanitize external data, or aggregate candidate actions.

C.2.1 TOOL-FILTER DEFENSE AND IMPLEMENTATION

  • Tool Filter first asks the model to select relevant tools, then exposes only the validated selections to the agent.The evaluation starts the second stage from a fresh conversation with the filtered tool list embedded in the user message to avoid a vLLM HTTP 400 error caused by reusing the assistant-ending history.Source: §C.2.1 · pp. 18–19

C.2.2 SPOTLIGHTING WITH DELIMITING DEFENSE

  • Spotlighting with delimiting inserts separators around instructions and tool-use boundaries, but may increase successive tool-call blocks and parsing errors.The reported errors on Mistral-Small-3.1 are associated with lower benign utility.Source: §C.2.2 · p. 19

C.2.3 PROMPTARMOR

  • PromptArmor uses an off-the-shelf LLM to detect and remove potentially injected instructions from external data before forwarding the sanitized input.It is integrated as a preprocessing defense without changing or fine-tuning the underlying agent.Source: §C.2.3 · p. 19

C.2.4 SECINFER

  • SecInfer generates five candidate actions from diverse prompts and uses target-task-guided aggregation to select the action most consistent with the original instruction.The evaluation uses temperature 0.7 and the target agent’s underlying LLM at each decision step.Source: §C.2.4 · p. 19

C.2.5 CAMEL DEFENSE

CaMeL executes a program derived from the trusted request while tracking provenance and capabilities, and quarantines untrusted tool outputs. Its multi-stage execution adds substantial runtime overhead.

C.2.5 CAMEL DEFENSE

  • CaMeL has a privileged LLM translate the trusted request into a Python-like program, executed by a restricted interpreter with provenance and capability metadata propagated across values and calls.Potentially untrusted tool outputs pass through a quarantined LLM before becoming structured values.Source: §C.2.5 · p. 19
  • Program synthesis, restricted interpretation, error-driven regeneration, and quarantined-LLM processing make CaMeL significantly slower than lightweight prompt- or filtering-based defenses.The evaluation uses the same backbone model for privileged and quarantined LLMs.Source: §C.2.5 · p. 19
  • Table 4 reports AgentDojo results for Mistral-Small-3.1-24B-Instruct-2503, including benign utility, utility under attack, and overall ASR over 609 non-ambiguous pairs.Results are averaged over six attack strategies; higher utility and lower ASR are better.Source: Table 4 · p. 20

C.3 ADDITIONAL EXPERIMENT RESULTS

On Mistral-Small-3.1, ToolFence reports low attack success alongside retained utility, including for within-tool attacks. Its relative security behavior remains consistent across four AgentDojo domains, despite differing absolute ASR.

C.3 ADDITIONAL EXPERIMENT RESULTS

  • 0.13% overall ASR, 0.05% cross-tool ASR, and 0.60% within-tool ASR accompany 55.80% clean utility and 52.30% utility under attack for ToolFence.The comparison identifies ToolFence as having the most favorable security–utility trade-off in this evaluation.Source: §C.3 · p. 20
  • Tool Filter reduces cross-tool ASR to 0.39% but retains 7.69% within-tool ASR, while CaMeL’s strong security comes with a utility penalty.The results indicate that restricting tool availability alone does not adequately constrain malicious parameter substitution.Source: §C.3 · p. 20
  • Across Workspace, Slack, Travel, and Banking, ToolFence achieves consistently low ASR although absolute attack success varies by suite.Banking and Slack are generally more challenging because they include consequential state-changing operations and multi-step data dependencies.Source: §C.3 · p. 20

Related papers

Loading 2609.37196v1…