PaperlayerView exact revision on arXiv

Source-linked AI summary

Ghost in the Agent: Redefining Information Flow Tracking for LLM Agents

Yuandao Cai, Wensheng Tang, Cheng Wen, Shengchao Qin

arXiv:2604.23374v1cs.CR

TL;DR

LLM agents can transform or use untrusted content while selecting privileged actions, exposing a gap in traditional deterministic taint tracking. NeuroTaint audits execution traces offline with persistent provenance, semantic evidence, and causal analysis, and substantially outperforms FIDES across the paper’s provenance-focused evaluation while remaining effective on supplementary benchmarks.

  • Problem

    Traditional taint analysis does not capture paraphrased content, implicit decision influence, or cross-session provenance in LLM agents.

  • Method

    NeuroTaint reconstructs source-to-sink provenance offline using a persistent DCPG and sink-time analyzers for semantic content propagation and implicit control influence.

  • Results

    NeuroTaint substantially outperforms FIDES on TaintBench and remains effective on InjecAgent and ToolEmu.

  • Takeaways & Limitations

    Provenance auditing should distinguish tainted information flow from generic unsafe-action classification and preserve lineage across memory and session boundaries.

  • Takeaways & Limitations

    Ten false negatives arise when paraphrase or delayed memory rehydration attenuates semantic evidence below the operating threshold.

Abstract

from arXiv · show

Autonomous Large Language Model (LLM) agents are increasingly deployed to conduct complex tasks by interacting with external tools, APIs, and memory stores. However, processing untrusted external data exposes these agents to severe security threats, such as indirect prompt injection and unauthorized tool execution. Securing these systems requires effective information flow tracking. Yet, traditional taint analysis that is designed for program memory states fundamentally fails when applied to LLMs, where data propagation is governed by probabilistic natural language reasoning. In this paper, we present NeuroTaint, the first comprehensive taint tracking framework tailored for the unique information flow characteristics of LLM agents. Our key insight is that taint propagation in LLM agents must be understood not only as explicit content transfer, but also as semantic transformation, causal influence on decisions, and cross-session persistence through memory. NeuroTaint therefore audits execution traces offline to reconstruct provenance from untrusted sources to privileged sinks using semantic evidence, causal reasoning, and persistent context tracking, rather than relying on exact string matches or pre-defined source-sink paths alone. Extensive evaluation using TaintBench, our 400-scenario benchmark spanning 20 real-world agent frameworks, shows that NeuroTaint substantially outperforms FIDES, an information-flow-control (IFC)-style baseline for LLM agents, in source-sink propagation detection. We further show that NeuroTaint remains effective on established agent-security benchmarks, including InjecAgent and ToolEmu, while operating offline with modest additional auditing cost.

1 Introduction

LLM agents expand the attack surface by turning untrusted external content into inputs for privileged actions, while traditional taint analysis cannot reliably track paraphrased, implicit, or cross-session influence. NeuroTaint addresses this gap with offline provenance auditing across multiple propagation modes.

  • Untrusted webpages, emails, and databases can steer agents toward unauthorized actions such as forwarding sensitive documents or executing tools.
  • Traditional taint trackers assume deterministic program states, but LLM agents rewrite, summarize, translate, and implicitly absorb untrusted text.
  • NeuroTaint treats source-to-sink tracking as provenance reconstruction across explicit content, implicit control influence, and asynchronous memory reuse.
  • Its offline workflow ingests execution logs and uses a DCPG-backed provenance backbone with ordered sink-time analyzers.
  • LLMs can suppress some unsafe propagations or review flagged flows, but they complement rather than replace provenance-aware auditing.
  • NeuroTaint substantially outperforms FIDES on TaintBench and remains effective on InjecAgent and ToolEmu.

2 Motivating Examples

Three motivating cases show why agent information-flow auditing must handle semantic rewriting, decision influence, and memory-mediated reuse. NeuroTaint combines semantic, causal, and persistent provenance evidence to distinguish these flows from merely correlated tool paths.

  • TaintBench instantiates three information-flow cases across real open-source agent frameworks: explicit content, implicit control, and asynchronous provenance reuse.
  • E1 — Semantic Paraphrasing: NeuroTaint uses meaning-preserving semantic similarity to link rewritten sink content to its source payload.
  • E1 — Semantic Paraphrasing: Semantic paraphrasing can preserve malicious meaning even when exact strings disappear, making path-level detection insufficient.
  • E2 — Logical Hijacking: Implicit control attacks alter which sink is invoked without copying source content, so detection requires counterfactual control-influence evidence.
  • E3 — Cross-Session Memory Propagation: Asynchronous reuse splits propagation across sessions, requiring taint persistence through memory and reconstruction of the later sink connection.
  • Summary and Design Implications: The motivating examples indicate that no single existing technique handles all three flow classes.

3 Threat Model and Problem Definition

NeuroTaint models agents as event-driven systems and defines source-to-sink propagation through three distinct flow classes. Its problem statement requires reconstructing lineage across tools, memory boundaries, and sessions, then explaining detected paths.

  • Threat Model: The threat model gives web-level attackers control over external content but not system prompts, model weights, or execution environments.
  • Threat Model: The defender deploys NeuroTaint as an auditing layer with execution logs, reasoning traces, and configurable source and sink policies.
  • Threat Model: The paper excludes multimodal attacks and semantic steganography from its considered attacker capabilities.
  • LLM Agent Execution Model: An agent execution is modeled as a sequence of tool events recording each tool, its arguments, and its returned value.
  • Problem Definition: Sources return potentially attacker-controlled content, while sinks invoke security-relevant effects such as email sending, code execution, or HTTP requests.
  • Three Classes of Information Flow: The three flow classes distinguish transformed content, decision influence without content transfer, and provenance crossing persistent-memory or session boundaries.
  • Problem Statement: NeuroTaint reconstructs lineage across tool calls and memory boundaries, detects propagation, and explains each event with an annotated provenance path.

4 NeuroTaint: Design

NeuroTaint builds a provenance graph that preserves taint lineage across memory and session boundaries, then performs sink-time checks for explicit semantic propagation and causal influence. Its hybrid tracker combines multiple evidence tiers, while behavioral probes improve implicit-flow recall at an extra LLM-call cost per sink.

  • Dynamic Context Provenance Graph: NeuroTaint constructs a DCPG that records tool and memory flows, persists taint labels across storage and sessions, and restores lineage for later sink audits.The graph is built incrementally from trace events and supports cross-session provenance restoration.
  • Sink-Time Auditing: Sink-time auditing combines recovered source lineage with explicit content checks and causal analysis, reporting propagation when either check succeeds.Retrieval restores lineage but does not itself produce a verdict.
  • Hybrid Semantic Tracker: The tracker uses configurable thresholds for lexical overlap, semantic similarity, retrieval-conditioned semantic matching, and fragment coverage.Concrete threshold values are fixed in the evaluation setup rather than in the symbolic method description.
  • Hybrid Semantic Tracker: The four-stage explicit tracker progresses from exact canary matching through LCS, semantic embeddings, and multi-fragment comparison to handle increasingly transformed content.The cascade is ordered from cheaper, more precise signals to broader, more expensive ones; each tier addresses a distinct coverage gap.
  • Hybrid Semantic Tracker: Tier 1 achieves precision = 1.0 with zero false positives for verbatim exfiltration and can survive summarization because the canary is appended.Canary matching requires injecting a unique UUID into source content.
  • Sink-Driven Causal Analyzer: Behavioral probes add one extra LLM call per sink invocation, while ablations show that removing the causal analyzer substantially reduces recall on implicit-flow scenarios.The reported overhead is bounded in practice because sinks are sparse.

5 Evaluation

The evaluation compares NeuroTaint with FIDES across provenance-focused and supplementary unsafe-agent benchmarks, then analyzes detection stages, error boundaries, cost, threshold sensitivity, and model-assisted cascades.

  • Experimental setup: TaintBench fixes scenario-level propagation labels before comparison, while ToolEmu and InjecAgent use benchmark-native unsafe labels as supplementary transfer checks.TaintBench treats propagation-positive as the positive class rather than unsafe action.
  • RQ1: End-to-end detection: NeuroTaint at least matches FIDES on existing agent-security benchmarks, while both achieve 0.623 precision on ToolEmu’s filtered injection-like subset.These rows are not label-identical to TaintBench and should not be read as one leaderboard.
  • RQ1: End-to-end detection: NeuroTaint reaches Precision = 0.921, Recall = 0.935, and F1 = 0.928 on TaintBench, compared with FIDES at 0.505, 0.540, and 0.522.The benchmark contains 400 scenarios spanning the reported overall outcome.
  • RQ2: Detection mechanisms: Stage-level recall is strongest for canary exact match, string provenance, and Multi-fragment Coverage, while semantic explicit evidence recalls 75 of 85 cases.Sink-Driven Counterfactual Analysis recalls 37 of 40 cases, compared with FIDES’s 0.425 on that subset.
  • RQ3: Detection boundaries: NeuroTaint misses 13 propagation-positive scenarios and raises 16 false alarms, with false negatives concentrated in semantic attenuation and false positives near topical-overlap and prior-knowledge boundaries.FIDES produces 106 false positives and 92 false negatives across broader semantic, coverage, cross-session, and implicit-control settings.
  • RQ4: Audit cost: Offline auditing adds 0.25 s per execution unit on average, while sink-driven causal queries average 457 tokens when token traces are available.The audit runs after execution rather than on the agent’s interactive path.
  • Sensitivity analysis: Across local threshold sweeps, overall F1 remains stable, with the largest variation at the lower ends of string and implicit-string thresholds.Lower thresholds admit more false positives, whereas higher thresholds introduce more false negatives.
  • Model-assisted analysis: Stronger execution models reduce realized propagation recall without replacing provenance analysis; under claude-opus-4-6, NeuroTaint still reports F1 = 0.678 versus FIDES at 0.343.In reviewer cascades, stronger models improve unsafe-action decisions after NeuroTaint localizes candidate flows.

6 Related Work

NeuroTaint complements prompt-level defenses and runtime enforcement by auditing completed agent trajectories for provenance from untrusted sources to sensitive sinks. Related systems instead focus on preventing attacks, checking task alignment, fuzzing vulnerabilities, or enforcing policies during execution.

  • Prompt-level defenses reduce attack opportunities before or during execution, whereas NeuroTaint analyzes provenance in completed agent traces.
  • Runtime systems such as AgentFuzz, RTBAS, and FIDES seek to discover or prevent exploitable flows during execution, while NeuroTaint performs offline provenance auditing.
  • Task Shield checks action alignment, while memory-poisoning work demonstrates that attacker-controlled information can persist and be reused across turns.

7 Conclusion

NeuroTaint is presented as an offline provenance auditor that combines persistent lineage tracking with sink-time analyses for agent source-to-sink propagation. TaintBench provides a reproducible benchmark and baseline for future agent-security research.

  • NeuroTaint combines a persistent DCPG backbone with sink-time analyzers for explicit content propagation and implicit control influence.
  • The framework separates provenance auditing from generic unsafe-action classification in LLM agents.
  • The open-source release, including TaintBench, provides a reproducible baseline for future agent-security work.
  • Sources are tool outputs that may carry attacker-controlled data, while sinks are tool calls whose inputs can cause side effects when tainted.

B TaintBench Overview

TaintBench is designed specifically to evaluate source-to-sink provenance detection rather than generic unsafe actions. Its labels capture whether attacker-controlled content reaches or counterfactually influences benchmark-defined sinks.

  • TaintBench asks whether attacker-controlled content observed at a benchmark-defined source reaches or counterfactually influences a benchmark-defined sink.
  • Its labels are propagation labels, not unsafe-action labels.
  • The current release contains 400 scenarios across 20 real-world open-source LLM agent frameworks.

B.1 Framework Coverage

TaintBench covers diverse LLM-agent architectures and execution surfaces rather than variants of a single agent loop. Its framework set includes single-agent, multi-agent, graph-based, retrieval/memory, browser, and workflow architectures.

  • TaintBench includes 20 real-world open-source LLM agent frameworks spanning single-agent, multi-agent, graph-based, RAG/memory, browser, and workflow-agent architectures.
  • The framework coverage is intended to demonstrate diverse execution and source/sink surfaces rather than lightweight adoption differences.

B.2 Benchmark Label Semantics and Manual Validation

TaintBench defines propagation by provenance: attacker-controlled content must reach a sink payload or counterfactually influence a sink action. Its 400 scenarios use framework-specific instantiations of common templates and were manually validated before detector comparison.

  • Propagation-positive scenarios require attacker-controlled content to reach the sink payload or counterfactually influence the sink action.Co-occurrence of source and sink without source attribution is classified as non-propagating.
  • Two authors independently reviewed all 400 scenario templates and representative traces before computing detector metrics.Disagreements were resolved by checking whether source content was necessary for the sink payload or action, and labels were fixed before comparing detectors.
  • 400 scenarios span 20 frameworks, with each framework contributing 20 scenarios instantiated from common benchmark templates.Each instantiation maps the intended provenance behavior onto native tools, memory mechanisms, and orchestration APIs.
  • The benchmark is balanced between 200 propagation-positive and 200 non-propagating scenarios.The non-propagating cases stress source–sink co-occurrence, topical overlap, and prior-knowledge attribution without true propagation.

C Family-Level Result Breakdown

Table 10 reports propagation outcomes by evaluation family using a full TP/FP/FN/TN breakdown. The family-level analysis refines the benchmark’s design-time scenario buckets for outcome analysis.

  • Table 10 reports TP, FP, FN, and TN outcomes separately for each evaluation family.These families are used for outcome analysis rather than only benchmark construction.
  • The evaluation families refine the design-time buckets introduced in Table 9.Table 9 explains benchmark balance, while Table 10 divides those buckets into outcome-analysis families.
  • The family-level breakdown distinguishes benchmark composition from detector outcome reporting.This separation clarifies that the design-time table is not itself a detector result table.

D NeuroTaint + LLM Judge Cascade

The NeuroTaint-plus-LLM cascade sends NeuroTaint-surfaced candidate flows to a bounded judge that decides whether realized sink actions should be blocked. Its metrics are computed over all 400 TaintBench scenarios.

  • Cascade setup: 203 candidate source-to-sink flows are surfaced from 400 TaintBench scenarios before LLM judging.Only NeuroTaint-surfaced candidates are sent to the second-stage judge.
  • Judge comparison: The cascade evaluates four judges: gpt-4.1-mini, gpt-4.1, claude-sonnet-4-6, and claude-opus-4-6.These judges are compared under the majority-vote TaintBench setting used by the main paper.
  • Judge role: The LLM judge performs unsafe-action triage rather than independently deciding whether propagation occurred.The bounded review context includes recovered source content, sink payload, final output, NeuroTaint evidence, and a compact tool trace.
  • Judge criteria: The judge blocks cases when sourced content is exfiltrated, persisted, published, executed, or operationalized in a way that should be blocked.It treats the source as untrusted or policy-sensitive and focuses on the realized source content, sink payload, and side effect.
  • Benchmark scope: TaintBench covers 20 open-source projects, while separate catalogues enumerate the evaluated frameworks’ source and sink tools.The framework-coverage table also categorizes architectures such as single-agent, multi-agent, graph-based, and RAG/memory systems.
  • Metric interpretation: Cascade metrics are computed over all 400 TaintBench scenarios, with FN including missed positives and surfaced positives judged safe by the LLM.Table 11 defines TP as a propagation-positive scenario ultimately blocked as unsafe.
Loading 2604.23374v1…