PaperlayerView exact revision on arXiv

Source-linked AI summary

Extracting Knowledge from Tools in LLM Agents

Chuanchao Zang, Jianing Wang, Wenyu Chen, Xiangtao Meng, Li Wang, Xinyu Gao, Yingkai Dong, Zheng Li, Shanqing Guo

arXiv:2608.30288v1cs.CR

TL;DR

LLM agents’ knowledge-based tools expose source content through mediated access, raising the question of whether public interactions can reconstruct hidden sources. The paper proposes ToolSiphon, a query-only attack using target alignment and source expansion, and reports substantial recovery across tools and datasets, with effectiveness also under limited tool information and representative defenses.

  • Problem

    Public agent interactions may progressively expose source content behind knowledge-based tools, enabling reconstruction despite the absence of direct source access.

  • Method

    ToolSiphon uses target alignment and source expansion signals to guide query-only extraction through public agent interactions.

  • Results

    74.3% of source records are recovered on average with non-target tool descriptions, with 83.2% textual recovery and 90.2% semantic similarity across three tool types and six datasets.

  • Takeaways & Limitations

    Agent-mediated access can support progressive reconstruction of hidden knowledge sources across heterogeneous tools, defenses, and real-world agent-building platforms.

  • Takeaways & Limitations

    ToolSiphon is less effective when target and related tools have identical functionality but entirely disjoint data, and it does not model fine-grained permissions over individual records.

Abstract

from arXiv · show

LLM agents commonly use knowledge-based tools and access their underlying files, databases, and search indexes through tool invocation. This integration improves agents' ability to provide domain-specific services but also introduces the risk of tool-mediated knowledge extraction: source content exposed to an agent for legitimate responses may be progressively recovered from its outputs, enabling reconstruction of the knowledge source behind a target tool. This paper systematically investigates this risk and identifies two challenges introduced by tool invocation: tool-selection uncertainty, where an agent may invoke a competing tool instead of the target tool, and tool-argument compression, where fine-grained query information may be lost when the agent generates tool arguments. To tackle these challenges, we propose ToolSiphon, a query-only extraction attack that introduces two complementary signals: a target-discriminative signal, implemented through Tool Contrastive Analysis, to steer queries toward the target tool; and a response-grounded factual signal, implemented through Evidence Chained Feedback, to mitigate argument compression and progressively expand extraction coverage. Across three types of knowledge-based tools and six domain-specific datasets, ToolSiphon recovers 74.3% of source records on average when coarse-grained information about non-target tools is available, with 83.2% textual recovery and 90.2% semantic similarity. Even without such information, it recovers 66.3% of source records. ToolSiphon also remains effective against representative defenses and on three real-world agent platforms.

1 Introduction

LLM agents’ knowledge-based tools create a public-interface attack surface through which adversaries may progressively recover underlying sources. ToolSiphon addresses tool-selection uncertainty and tool-argument compression, achieving substantial reconstruction across diverse tools and datasets.

  • 1 Introduction: Knowledge-based tools expose files, retrieval indexes, databases, search engines, and curated knowledge bases to agents through information-returning interfaces.The paper focuses on these tools rather than action tools that modify external state.
  • 1 Introduction: Source content used for legitimate answers may be recovered through agent responses, creating intellectual-property and business risks.Recovered clinical guidelines or proprietary research reports could support competing applications.
  • 1 Introduction: Tool-mediated extraction introduces tool-selection uncertainty because agents may invoke a wrong or related tool when capabilities overlap.The agent’s query must pass through tool selection before reaching the target source.
  • 1 Introduction: 74.3% of source records are recovered on average with non-target tool descriptions, alongside 83.2% textual recovery and 90.2% semantic similarity.Without those descriptions, the attack recovers 66.3% of source records on average.
  • 1 Introduction: ToolSiphon is a query-only attack that combines target alignment with source expansion to reconstruct content behind a target tool.Its signals use target-specific functional phrases and compact evidence from prior answers to guide subsequent requests.
  • 1 Introduction: Evaluations span three knowledge-based tool types, six domain-specific datasets, multiple defenses, and three commercial agent-building platforms.The platforms are GPTs, Coze, and Dify.

2 Preliminaries and Related Work

The paper frames tool-mediated knowledge extraction as reconstructing a hidden source through repeated black-box interaction with an agent. It distinguishes this mediated setting from direct retrieval and formalizes reconstruction quality through coverage and fidelity.

  • 2 Preliminaries and Related Work: A knowledge-based tool connects an agent to a domain-specific source through a retrieval or query interface rather than direct user access.The agent invokes the tool and generates responses from returned information.
  • 2 Preliminaries and Related Work: Agent tool invocation consists of selecting a tool and converting a natural-language request into schema-conforming arguments.Argument generation may extract entities and rewrite the query.
  • 2 Preliminaries and Related Work: Prior RAG extraction attacks directly manipulate retrieval, whereas this problem must pass through agent-mediated tool selection and argument generation.This distinction motivates the paper’s focus on tool-selection uncertainty and information loss during argument generation.
  • 2.3 Problem Formulation: Tool-mediated knowledge extraction accumulates source-level information across repeated responses to reconstruct the underlying knowledge source.The attacker interacts through the agent rather than directly querying the source.
  • 2.3 Problem Formulation: The formal goal is to approximate a target source K ⋆ from query-response sequences without directly accessing K ⋆.The source contains items such as document chunks, database records, or indexed entries.
  • 2.3 Problem Formulation: Coverage measures distinct recovered content, while fidelity measures whether recovered items preserve source correctness and semantics.Under a limited interaction budget, the attacker seeks broad and faithful reconstruction.
  • 2.4 Threat Model: The threat model gives the attacker only public task-oriented queries and final responses, excluding prompts, traces, tool decisions, intermediate outputs, and backend knowledge.The attacker targets one tool among multiple knowledge-based tools and may or may not know coarse descriptions of competing tools.

3 Our ToolSiphon

ToolSiphon reconstructs a target knowledge source through public agent interactions using two complementary signals: target alignment and source expansion. Its iterative pipeline generates adaptive task-oriented queries, accumulates response-derived evidence, and aggregates responses into reusable source-level knowledge.

  • Overview: ToolSiphon is a query-only attack that reconstructs a target tool’s knowledge source by submitting task-oriented queries and observing only final agent responses.The attacker lacks access to system prompts, tool outputs, execution traces, and the underlying source.
  • Challenges: ToolSiphon addresses tool-selection uncertainty and tool-argument information compression, which can route queries to related tools or discard fine-grained query information.Related tools may return relevant information from an unintended source, while argument generation may compress extraction-specific details.
  • Target Alignment via Tool Contrastive Analysis: Tool Contrastive Analysis constructs a target-alignment signal by selecting functional characteristics that are more specific to the target than to competing tools.When non-target descriptions are unavailable, TCA generates shadow tools representing plausible competing capabilities.
  • Source Expansion via Evidence-Chained Feedback: Evidence-Chained Feedback maintains an ordered memory of compact response-derived evidence items and retrieves a moving window to guide subsequent exploration.It removes duplicate items, adjusts retrieval according to discovery gain, and prioritizes productive regions while reducing redundant exploration.
  • Iterative Reconstruction: The two signals are combined to generate adaptive queries that progressively expand reconstruction coverage from initial target-guided exploration to evidence-guided exploration.Responses are decomposed and aggregated at the record level because a response may contain multiple records or overlap with earlier recovery.

4 Evaluation

ToolSiphon is effective across tool types, datasets, agent settings, and deployment variations, while its two components address complementary extraction challenges. Performance remains strong under many conditions but declines with severe description noise, constrained arguments, larger relevant-tool pools, and source-scale expansion.

  • Overall effectiveness: 76.0% average KSR under A1 and 69.5% under A2 demonstrate strong reconstruction across six datasets and three tool types.Under A1, average KSR is 83.5% for RAG, 64.0% for database, and 80.5% for search-engine tools.
  • Component ablation: 68.7% EC, 80.0% CRR, 92.0% SS, and 71.8% KSR are achieved when TCA and ECF are combined.Removing both components reduces average EC and KSR to 16.7% and 9.7%; TCA improves alignment, while ECF supports broader exploration.
  • Evidence-item strategy: 60.0% average EC with ECF evidence items exceeds random-entity, mismatched-entity, and semantic-mutation strategies at 30.0%, 20.0%, and 20.0%.The results attribute the advantage to preserving contextually relevant evidence chains; semantic mutations can interfere with argument generation.
  • Robustness: ToolSiphon remains effective with incomplete tool descriptions, but severe perturbation lowers A1 EC/KSR from 71.7%/75.3% to 53.0%/55.3%.Under A2, the corresponding scores decrease from 65.3%/68.3% to 52.3%/56.7%.
  • Robustness: Shadow-tool generator choice has limited effect, with average EC ranging from 62.3% to 67.0% and KSR from 65.7% to 70.7%.Adding shadow tools improves performance up to approximately five, after which gains are limited; five tools yield 76.3% CRR, 89.3% SS, and 68.3% KSR.
  • Deployment factors: Average EC decreases by approximately 7.0 percentage points with five arguments or strict argument-type constraints.Increasing relevant tools also decreases EC and KSR in most cases, whereas adding irrelevant tools has limited impact.
  • Model and architecture variation: ToolSiphon remains effective across victim-side LLMs and agent architectures, although database-backed tools show greater variation.Reflection-based agents may achieve comparable or higher extraction performance, while attacker-side models produce stable results.

5 Discussion

The discussion shows that ToolSiphon systematically expands beyond ordinary usage while remaining bounded by users’ existing permissions. Its effectiveness depends on exploration coverage and persists under increasing tool-selection uncertainty.

  • Scope of the Risk: ToolSiphon targets accessible knowledge within existing user permissions rather than bypassing access controls.Permission-controlled tools bound extraction to the attacker’s authorized level.
  • Beyond Ordinary Usage: ToolSiphon chains exposed evidence to explore unrecovered source content, unlike legitimate queries driven by limited user needs.This distinction explains why ordinary interactions revisit narrower source regions.
  • Beyond Ordinary Usage: 84% KSR is achieved by ToolSiphon(A1) with 200 queries, versus 6%, 21%, and 29% for naive, HealthQA, and MedQA queries.Increasing legitimate queries to 500 and 1,000 does not close the gap.

6 Conclusion and Limitations

The paper identifies tool-mediated knowledge extraction as a systematic risk and presents ToolSiphon as a query-only attack addressing tool selection and argument-generation challenges. Its scope is limited by tool distinguishability, permissions, and the need for defenses that account for cumulative exposure.

  • Conclusion: ToolSiphon combines Tool Contrastive Analysis for target alignment with Evidence-Chained Feedback for progressive source exploration.The approach operates through public agent interactions and uses only query-based access.
  • Conclusion: Evaluations across heterogeneous tools, defenses, and real-world platforms show that agent-mediated access alone cannot prevent systematic knowledge-source reconstruction.The conclusion frames mediated access as insufficient protection by itself.
  • Limitations: ToolSiphon is less effective when target and related tools have identical functionality but entirely disjoint data.The method assumes partial overlap while retaining distinguishability.
  • Limitations: The study does not consider fine-grained permissions over individual tool records and calls for defenses addressing cumulative exposure across interaction trajectories.These are identified as limitations and future directions.

A From Agent Responses to Reconstructed Records

This section converts agent responses into reconstructed records through extraction and deduplication. The pipeline filters responses into self-contained records, removes duplicates, and achieves high agreement with human labels in the audited settings.

  • Post-processing: Post-processing applies record extraction and deduplication because responses may contain unrelated text and overlapping records.Duplicate information can otherwise enter the reconstructed dataset multiple times.
  • Record Extraction: Record extraction retains only content consistent with the target tool and forms each item as one self-contained record.The procedure excludes unrelated response text and avoids combining information across records.
  • Record Extraction: The extraction prompt removes conclusions, transitions, and explanations while preserving recurring structural fields belonging to the record format.It also forbids inferring or adding information absent from the response.
  • Post-processing: Deduplication flags record pairs above 0.9 semantic similarity and uses DeepSeek-V4-Flash to verify whether they describe the same underlying record.Confirmed duplicates are not inserted again.
  • Human Evaluation: Extraction accuracy ranges from 0.93 to 1.00, and every audited duplicate decision agrees with majority-voted human labels.The audit samples extraction and deduplication operations across six datasets and four attacks.

B Baseline Implementation Details

The baseline study adapts existing query-only extraction methods to the same public agent interface used by ToolSiphon. It includes ordinary, direct-request, iterative, and agent-adapted baselines under matched victim configurations.

  • Shared Setup: All baselines interact exclusively through the victim agent’s public interface using the same configurations as the main experiments.LLM-based methods use Gemini-3-Flash in the stated setup.
  • Naive: Naive generates ordinary task-oriented queries from the domain and coarse-grained target-tool functionality.Its prompt requests a specified number of queries for the target task.
  • Brute Force: Brute Force uses a fixed query that directly requests all source information associated with the target tool’s functionality.This baseline is extraction-oriented rather than task-oriented.
  • Iterative Baselines: RAG-Thief, IKEA, and MEXTRA iteratively use one response to guide the next query, while their procedures are adapted from direct-access settings.Only final agent responses are used after submission through the victim interface.
  • Jail-IKEA: Jail-IKEA is an agent-adapted baseline supplied with coarse-grained functionality for the target, relevant, and shadow tools.It serves as a stronger baseline under the corresponding attacker-information setting.

C Metric Details

The paper evaluates extraction using metrics that separately capture unique source coverage, lexical recovery, semantic fidelity, and overall reconstruction quality. Its KSR metric combines these dimensions while penalizing duplicates and irrelevant or hallucinated outputs.

  • Extraction Coverage: Extraction Coverage measures the fraction of target records recovered, using progressively stricter matching stages.Successful recovery requires matching the same underlying record while preserving its core information.
  • Matching: A maximum-weight one-to-one matching prevents repeated extraction of the same target record from increasing coverage.Each target and extracted record can appear in at most one matched pair.
  • Fidelity Metrics: Chunk Recovery Ratio measures lexical recovery, whereas Semantic Similarity measures whether extracted records preserve the meaning of their closest targets.ROUGE-L captures lexical overlap and token ordering; semantic similarity ranges from 0 to 1.
  • Knowledge-Source Reconstruction Score: KSR jointly accounts for textual recovery, semantic fidelity, source coverage, duplicate extraction, and irrelevant or hallucinated outputs.Its recall measures quality-adjusted source coverage, precision measures valid reconstructed outputs, and the final score is their harmonic mean.
  • Knowledge-Source Reconstruction Score: KSR ranges from 0 to 1, with higher values indicating broader and more faithful reconstruction with fewer duplicated, irrelevant, or hallucinated outputs.The score is defined as 0 when both KSR recall and KSR precision are 0.

D Tool Invocation Analysis

Tool invocation analysis measures whether extraction queries reach the target tool and whether they invoke it exclusively. In controlled evaluation, ToolSiphon achieves high target-routing rates, while real-world evaluation uses only public-interface responses and excludes internal traces.

  • Invocation Metrics: Target Invocation Rate measures the fraction of queries that invoke the target tool.It is computed from the set of tools invoked for each query.
  • Invocation Metrics: Exclusive Target Invocation Rate measures the fraction of queries for which the target tool is the only invoked tool.Together, TIR and ETIR analyze target routing and competing-tool invocation.
  • Evaluation Setting: TIR and ETIR are computed through controlled-environment instrumentation and are unavailable to attackers in real-world black-box settings.Attackers observe only final agent responses rather than tool-selection decisions or invocation traces.
  • Results: 98.0% and 97.2% are ToolSiphon’s average TIRs under A1 and A2, respectively.The corresponding ETIRs are 97.7% and 95.8%, substantially exceeding the strongest baselines.
  • Attack Design: ToolSiphon uses target-alignment and source-expansion signals to guide query generation and incorporate atomic evidence from prior responses.The system prompts instruct query generation to target relevant functionality and use extracted entities, identifiers, field names, and values.

G Real-World Audit of Access Controls in Knowledge-Based Tools

The audit examines whether knowledge-based tools publicly document or implement fine-grained access controls. Many tools lack explicit documented or code-level restrictions, but the study scopes extraction to information already permitted by supplied credentials.

  • Audit Scope: The audit defines fine-grained access control as document-, row-, field-, user-, role-, or tenant-level restrictions.Authentication that grants access to an entire tool does not count as fine-grained control.
  • Documentation-Level Audit: 6,835 tools were collected from MCP Server, REST API, and LlamaIndex ecosystems, including 4,626 identified as knowledge-based tools.GPT-5.1 was used to assess whether tool documentation explicitly specified fine-grained controls.
  • Documentation-Level Audit: 56.2% of knowledge-based tools—2,600 of 4,626—do not document explicit fine-grained access controls.The absence of documented controls does not establish that a deployment has no authorization.
  • Code-Level Audit: 76.2% of inspected Dify plugin tools—560 of 735—contain no explicit fine-grained access-control enforcement in their code.For these tools, extractable scope is primarily determined by the permissions associated with supplied credentials.
  • Scope Boundary: The audit does not show that ToolSiphon circumvents backend authorization; it studies extraction within an attacker’s already permitted access scope.Broad tool-level access remains common in the audited setting.

H Additional Access Control Study

The additional study tests extraction under strict hierarchical access by partitioning records into membership tiers and enforcing backend filtering. ToolSiphon recovers only records within the attacker’s authorized tier.

  • Study Design: Knowledge sources are partitioned into Normal, Plus, and Pro records at ratios of 50%, 30%, and 20%, respectively.Random partitioning keeps tier distributions comparable while identity-based authentication and backend filtering enforce hierarchical access.
  • Results: All nine unauthorized task–tier combinations yield EC and KSR scores of 0.This result indicates no recovery above the attacker’s membership level.
  • Results: Across 18 authorized combinations, average EC is 0.686 and average KSR is 0.702.Authorized EC ranges from 0.51 to 0.86, while authorized KSR ranges from 0.54 to 0.87.
  • Results: Authorized-scope performance is strongest for RAG, followed by SE and DB.The reported RAG EC/KSR is 0.777/0.793, compared with 0.707/0.718 for SE and 0.575/0.5 for DB.

I Functionally Identical Tools with Disjoint Data

Functionally identical tools with disjoint data substantially impair source-specific extraction because the attacker cannot reliably distinguish which tool produced each record. Treating equivalent tools as joint extraction targets reduces this loss but cannot recover record provenance.

  • Experimental setup: ToolSiphon evaluates functionally identical related tools containing 200 non-overlapping records from the target domain under two replacement settings.Setting 1 replaces one related tool, while Setting 2 replaces both, using the same 200-query budget under A1 and A2.
  • Extraction effectiveness: 0.31 and 0.13 are the average KSR values under A1 in Settings 1 and 2, down from 0.75 in the original setting.The corresponding A1 KSR values decrease as more functionally identical tools are introduced.
  • Extraction effectiveness: 0.23 and 0.08 are the average KSR values under A2 in Settings 1 and 2, down from 0.68 in the original setting.The reduction occurs because records from other tools are mixed into the reconstruction and penalized as irrelevant data.
  • Scope boundary: No reliable black-box method is identified for reconstructing one specific tool when functionally identical tools provide little evidence about data provenance.An attacker can retain records from equivalent tools when reproducing functionality, but this does not recover individual-record provenance or solve source-specific extraction.
Loading 2608.30288v1…