PaperlayerView exact revision on arXiv

Source-linked AI summary

AgenTRIM: Tool Risk Mitigation for Agentic AI

Roy Betser, Amit Giloni, Shamik Bose, Sindhu Padakandla, Chiara Picardi, Lidor Erez, Roman Vainshtein

arXiv:2601.12449v2cs.CRcs.AI

TL;DR

Tool-enabled agents face security risks from excessive or insufficient permissions, especially when external content or unreliable descriptions influence tool use. AgenTRIM reconstructs and verifies tool interfaces offline and adaptively controls per-step access online without modifying agent reasoning. Across indirect prompt injection, description-based risks, and policy enforcement, it improves the safety-utility trade-off while maintaining task performance.

  • Problem

    External tools expand agent capability and attack surface, while existing tool permissions can produce indirect prompt injection, misuse, excessive agency, or insufficient agency.

  • Method

    AgenTRIM reconstructs and verifies the tool inventory from code, controlled executions, and traces, then dynamically controls tool access at each step using task status.

  • Results

    Across indirect prompt injection, description-based tool risks, and policy enforcement, AgenTRIM achieves low attack success, high utility, and improved safety-utility trade-offs.

  • Takeaways & Limitations

    Controlling tool access rather than modifying agent reasoning provides a practical foundation for safer LLM-based agents.

  • Takeaways & Limitations

    Online orchestration depends on control metadata whose quality and domain-specific annotations affect the safety-utility trade-off.

Abstract

from arXiv · show

AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risks such as indirect prompt injection and tool misuse. We characterize these failures as unbalanced tool-driven agency. Agents may retain unnecessary permissions (excessive agency) or fail to invoke required tools (insufficient agency), amplifying the attack surface and reducing performance. We introduce AgenTRIM, a framework for detecting and mitigating tool-driven agency risks without altering an agent's internal reasoning. AgenTRIM addresses these risks through complementary offline and online phases. Offline, AgenTRIM reconstructs and verifies the agent's tool interface from code and execution traces. At runtime, it enforces per-step least-privilege tool access through adaptive filtering and status-aware validation of tool calls. Evaluating on the AgentDojo benchmark, AgenTRIM substantially reduces attack success while maintaining high task performance. Additional experiments show robustness to description-based attacks and effective enforcement of explicit safety policies. Together, these results show that AgenTRIM provides a practical, capability-preserving approach to safer tool use in LLM-based agents.

1 Introduction

AgenTRIM frames tool-related failures as unbalanced tool-driven agency and addresses them by reconstructing tool interfaces offline, then dynamically controlling tool access at runtime while preserving agent reasoning.

  • Motivation: Broader tool access expands capability but also increases exposure to indirect prompt injection and unsafe or unintended tool use.Malicious payloads can originate in tool descriptions, external content, or tool outputs.
  • Motivation: Tool-driven agency risks often arise from permitted tools and their integration into decisions rather than from the LLM’s reasoning itself.The framework targets both excessive and insufficient agency by dynamically constraining access while preserving legitimate task autonomy.
  • AgenTRIM framework: AgenTRIM reconstructs and verifies the agent’s tool interface offline, then dynamically shapes available capabilities at each execution step online.The offline stage extracts candidate definitions, validates existence through controlled executions, and analyzes traces to verify behavior and descriptions.
  • AgenTRIM framework: The framework mitigates excessive and insufficient agency while preserving the agent’s original reasoning and logic.Stepwise capability control limits unnecessary exposure without modifying the agent’s internal decision process.
  • Evaluation: On AgentDojo and additional description-based tests, AgenTRIM reduces attack success while maintaining high utility and enforcing explicit safety policies.The evaluation covers erroneous tool descriptions, malicious MCP-server descriptions, and policy enforcement beyond prompt injection.

2 Background and Related Work

Prior defenses mainly impose runtime restrictions for specific tool-use threats, while AGENTRIM targets broader capability exposure and incomplete or unreliable tool inventories.

  • Existing defenses: Existing defenses use guardrails, policy checks, planning-execution separation, or graph heuristics to restrict runtime tool behavior.These approaches particularly focus on indirect prompt injection and individual unsafe tool calls.
  • AgenTRIM’s distinction: AGENTRIM constrains available capabilities at each decision point based on task status while preserving the agent’s original reasoning process.This contrasts with approaches whose restrictions can cause otherwise solvable tasks to fail and reduce utility.
  • Limitations of prior work: Prior approaches commonly assume a fixed, complete, accurate, and faithfully described tool inventory.That assumption is problematic because inventories may be incomplete, deployment-specific, change over time, or contain erroneous or manipulated descriptions.

3 AGENTRIM

AGENTRIM combines execution-grounded offline tool verification with a runtime orchestrator that adaptively limits and validates tool access without changing the agent’s reasoning. Its pipeline reconstructs a verified inventory, exposes compatible tools per step, rejects unsafe calls, and tracks progress across restricted iterations.

  • Offline tool extractor: AGENTRIM first reconstructs a verified tool inventory by combining code analysis, controlled executions, trace analysis, metadata generation, and discovery of missed tools.Static extraction prioritizes coverage, while execution traces verify tool existence, behavior, descriptions, and observed inputs and outputs.
  • Offline tool extractor: Execution-grounded verification shifts tool extraction beyond static or description-based inference toward an interface based on observed agent behavior.
  • Online tool orchestrator: AGENTRIM dynamically exposes a task-dependent subset of the verified inventory, ensuring the exposed set has no greater modeled risk than the full tool set.The subset is derived from verified tools, proposed calls, and control metadata.
  • Online tool orchestrator: Adaptive filtering selects the largest compatible subset of proposed tools and, when tied, prefers the option permitting broader exposure before deferring incompatible calls.This compatibility-based exposure implicitly decomposes execution into multiple steps.
  • Online tool orchestrator: A deterministic trigger routes selected calls to an LLM judge using current status, while rejected calls are not executed and are recorded to prevent repeated attempts.
  • Online tool orchestrator: The status manager observes each restricted loop, constructs the next status from the query, executed calls, and outputs, and terminates or enables another loop without intervening in reasoning.This separates status tracking from the agent’s decision-making while supporting stepwise control.

4 Evaluation

AGENTRIM is evaluated across indirect prompt injection, description-based risks, tool extraction, ablations, and policy enforcement. It consistently improves the safety–utility trade-off through execution-grounded extraction and stepwise runtime control.

  • Offline extractor: The extractor achieves zero fabrication and near-zero miss rates across 500 ReAct configurations, with perfect performance on external agents.Static analysis provides coverage, while trace-based validation is critical for recall and reducing fabrication.
  • AgentDojo evaluation: AGENTRIM achieves the lowest or near-lowest ASR while maintaining competitive utility across evaluated AgentDojo settings.It offers the best overall security–utility trade-off without larger standard deviations than baseline methods.
  • Comparison with leading defenses: AGENTRIM retains high utility, has the smallest utility drop under attack, and incurs approximately 1.85× latency versus roughly 3–9× for CAMEL and AgentArmor.It is also reported as the only method to improve utility over the baseline.
  • Description-based risks: Execution-grounded extraction reduces attack success from manipulative descriptions, reaches 0% ASR for shadow attacks, and remains stable as erroneous descriptions accumulate.The extractor validates tool functionality rather than trusting descriptions.
  • Policy integration: AGENTRIM achieves near-perfect policy compliance with safety-tool availability, reporting F1 = 0.995, PBR = 0.0, and functional utility F1 = 0.981.When safety tools are unavailable, it withholds policy-sensitive functional tools rather than permitting policy breaches.
  • Online orchestrator ablations: Removing adaptive filtering, status tracking, or the tool-call judge increases ASR, while removing status tracking also causes a large utility drop.These ablations support the need for stepwise filtering, status tracking, and call validation.
  • Online-component robustness: AGENTRIM remains robust when targeted attacks affect its status manager and tool-call judge, retaining lower ASR and higher utility than the baseline.The attacks slightly increase ASR and reduce utility for the baseline agent.

5 Conclusion

AGENTRIM treats tool-driven agency as the control target for mitigating tool-related risks. Its offline inventory reconstruction and online per-step access control improve the safety–utility trade-off without modifying the agent’s reasoning.

  • Framework: AGENTRIM reconstructs and verifies the agent’s tool inventory offline, then dynamically controls tool access at runtime.The offline phase uses controlled executions and trace analysis; the online phase shapes access at each step.
  • Conclusion: By limiting unnecessary exposure while preserving the agent’s original reasoning process, AGENTRIM supports safer tool use across prompt injection, description-based risks, and policy enforcement.The paper presents this as a practical foundation for safer LLM-based agents.

Limitations

AGENTRIM’s effectiveness and deployment cost depend on integration and control metadata. Runtime mediation also introduces measurable latency and cost overhead.

  • Integration boundary: Adapting AGENTRIM to new agentic frameworks may require one-time integration with framework-specific tracing or logging.The offline extractor is best run in an isolated environment, but this integration is not a runtime dependency.
  • Metadata dependence: Online orchestration depends on control metadata quality, and incorrect annotations may affect the safety–utility trade-off.Different applications may require domain-specific annotations for dependencies, policies, or task-specific risk.
  • Runtime overhead: AGENTRIM incurs approximately 1.85× latency and 2× cost relative to the baseline agent.The paper frames this overhead as a controllable trade-off for stronger safety guarantees.

Ethical considerations

The paper evaluates tool-misuse defenses under intentionally simulated adversarial conditions. Harmful-language examples are restricted to controlled measurement and are not intended to enable misuse.

  • Controlled evaluation: The evaluation intentionally simulates prompt-injection and other tool-related attacks to measure AGENTRIM’s safety.The paper frames these adversarial conditions as part of responsible AI evaluation.
  • Harmful-language examples: Appendix examples of harmful language are included solely to instantiate attacks, analyze failure modes, and strengthen defenses.The paper states that these examples are not intended to enable misuse.
  • Responsible deployment: AGENTRIM does not introduce new offensive capabilities, and adoption is encouraged alongside established safety and deployment practices.The paper presents the work as contributing to safer deployment of agentic systems.

Overview

The supplementary material provides reproducibility details, experimental analyses, attack examples, and implementation prompts, including a warning about harmful content.

  • The supplement extends the method description with full prompts and adds experimental settings, attack examples, ablations, and extended results.
  • Appendix B covers tool-extractor implementation details and ablations.
  • Appendix C covers tool-orchestrator implementation, ablations, and targeted attacks against its LLM-based components.
  • Appendices D and E provide MCP description-attack results and policy-integration experiment details.
  • The document includes the prompts used in AgenTRIM and warns that it contains harmful language and content examples.

A Reproducibility and LLM usage

The supplementary materials document AgenTRIM’s implementation, tool pool, deterministic code analysis, execution-based validation, and reproducibility practices.

  • The released code includes the extractor pipeline, an AgentDojo orchestrator adapter, and prompts for AgenTRIM’s LLM calls.
  • The authors state that LLMs supported the method, writing assistance, and code generation, while authors made experimental and interpretive decisions.
  • The evaluation tool pool includes LangChain, custom, Gmail, and MCP tools spanning web search, SQL, messaging, mathematics, and document functions.
  • Tools are used to evaluate extraction quality, but functionality is validated during tracing because some tools depend on outputs from other tools or multi-step interactions.
  • Code analysis deterministically builds an inventory by parsing ASTs, resolving imports and configurations, and detecting decorated functions, wrappers, subclasses, and registries.
  • Execution traces reveal invoked tools and their inputs and outputs, which are used to generate descriptions aligned with observed functionality.

B.3 Component ablations

Component ablations show that deterministic extraction and trace validation provide strong coverage and low fabrication, while targeted search addresses missing tools and AgenTRIM preserves the security–utility trade-off across evaluations.

  • Component ablations: Code with trace achieves the best extractor scores, 1.0/0.997, with fabrication 0.0; removing validation degrades recall and increases fabrication to 3.462.
  • Component ablations: None+Trace recall is 0.152, showing that omitting both code extraction and agent self-report harms tool coverage.
  • Search and discovery: Search and discovery reduces miss rate by approximately 20–25% across missing-tool conditions while maintaining zero fabrication; a small initial list yields about 14% error reduction.
  • Search and discovery: With a to-check list, search reaches approximately 95 recall and accuracy, recovering nearly all hidden tools with high precision and no fabrications.
  • LLM ablations: Across models, precision is 1.00 and fabrication is 0; recall ranges from 0.99 for GPT–4o to 0.93 for Cohere.
  • Detailed results: AgenTRIM achieves the lowest or near-lowest ASR while maintaining competitive utility, producing the best overall security-utility trade-off.
  • Competitor results: In the reported setup, AgenTRIM runs at 8.84s/task versus 4.92s/task for the baseline, yielding L ≈1.8×.
  • Competitor results: On the reproduced comparison, MELON has ASR = 1.05%, Progent 5.7%, and CaMeL 0%, with utility values also reported for each method.

C.5 Control metadata ablation

Control metadata determines how AgenTRIM balances attack resistance and utility: selective joint retrieval with isolated actions performs best, while uniform exposure strategies create clear trade-offs.

  • Metadata schemes: The main scheme exposes retrieval tools jointly and environment-modifying tools in isolated steps, alongside broadly permissive, fully isolated, and inverted alternatives.
  • Metadata schemes: The joint-retrieval / isolated-action scheme achieves the best balance, while broad exposure raises ASR and isolating all tools substantially reduces utility.
  • Exposure trade-off: As more tools receive isolated exposure, ASR decreases but utility also declines; the main operating point lies near the curve’s knee.
  • Domain-specific metadata: Manual metadata schemes reduce Slack ASR from 12% to 7.1%, 8.3%, and 10%, but increase ASR elsewhere and leave overall ASR similar.
  • Domain-specific metadata: The results support optimizing control metadata by domain and deployment rather than applying uniformly permissive or restrictive exposure.

D.1 MCP Preference Manipulation Attacks (MPMA)

The MPMA and tool-shadow experiments test whether manipulated MCP tool descriptions can bias tool selection or induce malicious tool chaining, and whether AGENTRIM’s extracted descriptions mitigate these attacks.

  • MCP Preference Manipulation Attacks (MPMA): MPMA manipulates malicious tool names and descriptions with authoritative, emotional, subliminal, and exaggeration-based advertising strategies.The evaluated functionalities are time, weather, cryptocurrency news, and Wikipedia summaries.
  • MCP Preference Manipulation Attacks (MPMA): The MPMA setup places one malicious and five benign tools with the same functionality in the agent’s tool bank, allowing direct comparison of selection behavior.The malicious tool may access prohibited files, while the experiment focuses on whether its manipulated description biases selection.
  • MCP Preference Manipulation Attacks (MPMA): Baseline agents show high MPMA attack success because manipulated descriptions influence tool selection, whereas AGENTRIM’s straightforward descriptions lower attack success across all functionalities.The experiment averages attack success across four advertising strategies; six tools share each functionality, making uniform random selection approximately 16.66%.
  • Tool-shadow attacks: Tool-shadow attacks embed instructions in benign tool descriptions to trigger unrelated tool calls or other malicious activity.Examples instruct factorization or multiplication tools to invoke web search or Wikipedia scraping for harmful content.
  • Tool-shadow attacks: AGENTRIM removes spurious instructions from tool descriptions and achieves 0% attack success across Add, Subtract, Multiply, Fibonacci, and Factorization shadow attacks.The revised descriptions are based on tool input and output structures, preventing the malicious calls shown in the baseline.

E Policy integration experiment details

The policy-integration experiment evaluates whether AGENTRIM can require designated safety tools alongside policy-sensitive functional tools while measuring utility and compliance under different tool-access configurations.

  • Evaluation objective: The setup isolates insufficient-agency cases in which required safety tools are missing or unavailable and quantifies the trade-off between utility and policy compliance.Functional utility is evaluated against intended tools, while safety usage is evaluated against the active policy.
  • Policy and tool setup: The experiment defines three safety tools: web_search_filter, data_leakage, and content_verifier.They sanitize outbound web queries, detect database leakage or PII risks, and verify generated content, respectively.
  • Policy and tool setup: Each policy permits a functional tool only when its corresponding safety tool is available and invoked.The specified mappings are web_search with web_search_filter, read_database with data_leakage, and generate_image with content_verifier.
  • Experimental protocol: The evaluation generates 1k balanced queries, each designed to trigger at least one known functional tool invocation.Queries are distributed evenly across the functional tools to provide balanced coverage.
  • Experimental protocol: Four configurations compare baseline and AGENTRIM behavior with or without access to required safety tools.The recorded tool sequences support separate evaluation of functional-tool correctness and policy-compliant safety-tool usage.
Loading 2601.12449v2…