PaperlayerView exact revision on arXiv

Source-linked AI summary

From Storage to Steering: Memory Control Flow Attacks on LLM Agents

Zhenlin Xu, Xiaogang Zhu, Yu Yao, Minhui Xue, Yiliao Song

arXiv:2603.15125v3cs.CR

TL;DR

Existing security analyses often overlook how persistent memory shapes LLM-agent control flows across tasks. The paper identifies MCFA, develops MEMFLOW to audit it systematically, and finds widespread vulnerability across models and tool frameworks, while noting that its main evaluation does not test common production-style mitigations.

  • Problem

    Existing analyses treat agent control flows as ephemeral sessions, overlooking memory’s persistent influence on cross-task tool selection and ordering.

  • Method

    The paper defines MCFA and uses MEMFLOW to audit memory injection and subsequent control-flow deviations across heterogeneous tasks and long interaction horizons.

  • Results

    Over 90% of evaluated trials are vulnerable to MCFA across three LLMs and tools from LangChain and LlamaIndex.

  • Takeaways & Limitations

    MCFA is prevalent and persistent, creating critical security risks for memory-augmented LLM agents.

  • Takeaways & Limitations

    The main evaluation characterizes vulnerabilities but does not test whether common production-style mitigations reduce MCFA.

Abstract

from arXiv · show

Modern agentic systems allow Large Language Model (LLM) agents to tackle complex tasks through extensive tool usage, forming structured control flows of tool selection and execution. Existing security analyses often treat these control flows as ephemeral, one-off sessions, overlooking the persistent influence of memory. This paper identifies a new threat from Memory Control Flow Attacks (MCFA) that memory can dominate the control flow, forcing unintended tool usage even against explicit user instructions and inducing persistent behavioral deviations across tasks. To understand the impact of this vulnerability, we further design MEMFLOW, an automated evaluation framework that systematically identifies and quantifies MCFA across heterogeneous tasks and long interaction horizons. To evaluate MEMFLOW, we attack state-of-the-art LLMs, including GPT-5 mini, Claude Sonnet 4.5 and Gemini 2.5 Flash on real-world tools from two major LLM agent development frameworks, LangChain and LlamaIndex. The results show that in general over 90% of trials are vulnerable to MCFA even under strict safety constraints, highlighting critical security risks that demand immediate attention.

1 Introduction

LLM agents’ tool calls form control flows, but existing analyses often overlook how persistent memory shapes those flows across tasks. The paper identifies MCFA and introduces MEMFLOW to evaluate its prevalence and persistence.

  • Motivation: Existing security analyses largely treat control flows as one-off sessions, overlooking memory’s role in shaping cross-task trajectories.Agent behavior can follow a recurring Read[memory] →Plan[control flow] →Execute[control flow] →Write[memory] loop.
  • Threat: Memory Control Flow Attacks (MCFA) use action-oriented policies stored in memory to force unintended tool usage despite explicit user instructions.Unlike traditional control-flow attacks limited to one session, MCFA can induce persistent behavioral deviations across tasks.
  • Framework: MEMFLOW systematically identifies and quantifies MCFA across heterogeneous tasks and long interaction horizons.Its components include an Agent Runner, Attack Generator, and Control-Flow Auditor.
  • Results: Over 90% of evaluated trials are vulnerable to MCFA across GPT-5 mini, Claude Sonnet 4.5, Gemini 2.5 Flash, LangChain, and LlamaIndex.Tool selection is corrupted by 91.7∼100% and tool ordering by 52.8∼69.4% within a single session; effects persist across tasks in 97.2∼100% of cases.
  • Contributions: The paper contributes a systematic security analysis, two deviation types, three persistence-related characteristics, MEMFLOW, and large-scale evaluation.The evaluation covers three state-of-the-art LLMs and real-world tools from LangChain and LlamaIndex.

2 Related Work

Prior work studies control-flow attacks and memory attacks largely separately. This paper addresses the gap by treating poisoned memory as a persistent control input that can alter observable tool selection and ordering across tasks.

  • Control-Flow Attacks: Control-flow attack studies target tool-call traces, including tool selection, invocation sequences, prompt injection, and tool-mediated manipulation.Examples include metadata manipulation, obfuscated prompts, and untrusted tool outputs.
  • Memory Attacks: Memory-attack studies examine poisoned knowledge, self-reinforcing injections, reasoning backdoors, privacy leakage, and defenses for corrupted memory.These studies focus on memory integrity, retrieved content, or interaction data rather than control-flow traces.
  • Research Gap: Prior analyses overlook how retrieved memory alters observable tool selection and ordering across tasks.The paper frames poisoned memory as a write-once, read-many control signal that can induce cross-task control-flow deviations.

3 Methodology

The methodology models memory-augmented agents, defines MCFA as memory-induced control-flow deviation, and audits attacks through isolated tool traces. MEMFLOW standardizes injection, execution, and evaluation across agent configurations.

  • Security Setting: The security objective is control-flow integrity: whether attacker-influenced memory biases tool selection and invocation order.The observable control flow is used as a proxy rather than evaluating output correctness.
  • Threat Model: The threat model grants an external adversary only ordinary interaction, allowing attacker-influenced content to enter memory without privileged system or tool access.The adversary cannot directly access system prompts, tool implementations, or the memory store.
  • Agent Model: A memory-augmented agent is modeled with an action-generation function, tools, persistent memory, and retrieval and update functions.Retrieval injects stored information into planning, while updating determines what persists for future tasks.
  • MCFA Definition: MCFA occurs when an injected adversarial policy is later retrieved and changes a benign task’s tool trace against the safety policy.A memory write alone is insufficient: attack success requires subsequent memory-induced control-flow deviation.
  • Attack Families: The evaluation covers OVERRIDE, ORDER, M-SCOPE, PERSISTENCE, and RESISTANCE, distinguishing tool-choice and workflow deviations from persistence properties.The latter three characterize cross-task propagation, long-horizon persistence, and relapse after correction.
  • Auditing: MEMFLOW performs injection followed by isolated benign-task evaluation, recording tool traces and computing ASR or family-specific metrics.Short-term history is reset so deviations can be attributed to retrieved long-term memory.
  • MEMFLOW: MEMFLOW standardizes agent execution, attack generation, and control-flow auditing across models, memory mechanisms, and tool frameworks.Its Agent Runner executes and logs traces, its Attack Generator creates trials, and its Auditor measures ISR, ASR, and related metrics.

4 Experiments

Across matched controls, malicious memory drives substantial control-flow deviations in tool selection and execution, including tool hijacking and workflow reordering. These effects generalize across tasks, persist over long horizons, resist textual correction, remain strong under sensitivity variations, and are only partially reduced by RBMS.

  • Main results: 0% ASR occurs for both clean-baseline and benign-memory controls across all five attack families, while malicious memory induces substantial control-flow deviations.The matched controls isolate malicious memory as the source of the deviations rather than retrieval alone.
  • Main results: MCFA diverts control flow through tool hijacking (Override) and workflow hijacking (Order).Override targets tool selection, whereas Order affects execution order.
  • RQ1: Control-flow hijacking: 97.2%–100% ASR occurs for Override under malicious memory, compared with 0% for both matched controls across models and frameworks.Malicious memory strongly biases selection toward the risky tool even when the default configuration favors the safe one.
  • RQ1: Control-flow hijacking: 52.8% to 69.4% Order ASR occurs under malicious memory, while both matched controls remain at 0%, including when success requires breaking the safe dependency chain.The attack therefore affects execution order as well as tool choice.
  • RQ2: Cross-task propagation: 97.2%–100% M-Scope ASR across frameworks and LLM backends shows that injected policies generalize beyond the original task template and propagate through retrieved memory.M-Scope remains high under the Isolated regime, supporting retrieval-driven propagation rather than conversational carryover.
  • RQ3–RQ4: Persistence and resistance: 100% Persistence ASR occurs across evaluated LLMs and frameworks under malicious memory, and textual correction fails to prevent later relapse when subsequent tasks trigger retrieval.The effect persists without re-injection, while post-hoc correction does not overwrite the malicious memory.
  • Sensitivity and realism: 91.7% ASR remains for GPT-5 mini and 94.4% for Claude 4.5 after adding 100 benign records, while summarization reduces ASR from 100% to 63.9% for GPT-5 mini and from 97.2% to 61.1% for Claude 4.5.Prompt and tool-label variations also fail to remove the attack effect; explicit harmful labeling provides only limited protection.
  • Defense evaluation: RBMS reduces Override ASR, with the hierarchy-patched variant D2 achieving the largest drop, but attacks remain non-zero and hierarchy non-compliance persists.The main evaluation’s limitation is that it initially characterized vulnerabilities without testing production-style mitigations.

5 Conclusion

The paper identifies MCFA as a persistent threat in which action-oriented memory can hijack tool selection and execution order across later benign tasks. It operationalizes this threat through observable control-flow integrity criteria and evaluates memory architectures, write paths, and persistence behaviors.

  • MCFA lets action-oriented policies stored in memory hijack tool selection and execution order across later benign tasks.The attack requires corrupted memories to be retrieved during normal tasks, producing systematic control-flow deviations.
  • RBMS separates user-related memory from system policies and safety information to reduce cross-boundary influence, but the paper evaluates whether segregation alone is sufficient.The memory architecture comparison includes a flat naive stream and role-based segregation.
  • The security goal is agent-level control-flow integrity: untrusted inputs should not systematically skew tool selection or invocation order through long-term memory.The evaluation focuses on observable control flow rather than output correctness, under both naive memory and RBMS-style segregation.
  • MCFA can enter through explicit preference writes, implicit summaries, or retrieval-cache writes, creating a “Write Once → Read Many → Stable Control Flow Deviation” chain.These write interfaces allow untrusted content to become persistent planning input.
  • The paper audits systematic tool-selection shifts, invocation-order changes, cross-task duration, and relapse after correction using corresponding benchmark metric families.These observable events support the Override, Order, M-Scope, Persistence, and Resistance evaluations.

A.6 Out-of-Scope

The out-of-scope analysis excludes attacks that do not use long-term memory to create cross-task control-flow deviations, as well as direct compromise of tools, systems, or infrastructure. The included protocol instead injects memory, refreshes short-term state, and audits subsequent tool traces.

  • Out-of-Scope: The paper excludes malicious tool supply-chain attacks, traditional single-turn prompt injection, system-level compromise, and unrelated side-channel or physical attacks.These scenarios target tools, one-turn outputs, trusted system components, or unrelated attack surfaces rather than the defined MCFA mechanism.
  • Evaluation Protocol: Each proof of concept uses injection followed by system re-initialization, then triggers a benign task while logging retrieved memory, planner decisions, and control flow.This separates persistent memory effects from residual chat context.
  • Override: Override targets selection of a specific, often higher-risk tool by causing risky_X to be called when safe_X is available and sufficient.Success requires the first Trigger-phase call to be risky_<X> despite safe_<X> availability.
  • Override Example: The email-drafting example stores a preference for risky_create_email_draft, refreshes the agent, and then retrieves that preference during a later benign drafting request.The logged Trigger phase invokes the preference tool with the risky tool preference and generates the requested draft.

B.3 Order: Workflow Reordering and Step-Skipping

Order attacks corrupt a workflow’s tool-invocation sequence by skipping, reordering, or prematurely executing steps. The example stores a fast_flow preference, then executes payment before the predefined safety checks and logging steps.

  • Order attacks cause sequence deviations such as skipped validation, premature execution, or reordering relative to a predefined safe workflow.The core signature is deviation from the required control-flow order.
  • The payment workflow’s safe order is payment_check_risk → payment_verify_user → payment_execute → payment_log.Order success is evaluated against this SAFE_ORDER sequence.
  • The stored fast_flow preference favors a faster payment flow that may skip safety checks and prioritize execution and logging.The preference is written before the agent is re-initialized and later asked to make a payment.
  • The Trigger trace invokes payment_execute directly and reports payment completion, demonstrating a deviation from SAFE_ORDER.The logged sequence omits the preceding risk-check and user-verification steps before execution.
  • Order hijack succeeds when the executed sequence contains any missing step, out-of-order step, or premature execution.This strict criterion treats each of these deviations from SAFE_ORDER as a successful hijack.

B.4 M-Scope: Cross-Task Propagation

M-Scope tests whether a persistent memory policy expands tool use across benign tasks with different templates or tool domains. Its success requires valid cross-task deviations caused by the same injected policy.

  • Attack goal and intuition: M-Scope injects an “always also do X” directive that expands tool use beyond the current task’s requirements.Its core signature is cross-task propagation during a task that does not warrant the additional tool use.
  • Success criterion: M-Scope succeeds when the same injected policy causes valid deviations across multiple benign tasks with different templates or tool domains.This criterion distinguishes cross-task propagation from a deviation confined to one task.
  • Evaluation procedure: The evaluation stores a persistent preference for using risky_tool, resets short-term context, and then tests unrelated tasks.The illustrated sequence includes a coffee-shop task followed by an email-drafting task after memory reinitialization.
  • Observed deviations: The attack example shows risky tools invoked for tasks that otherwise request ordinary mapping or email drafting assistance.The traces include risky_map_tool for finding a coffee shop and risky_create_email_draft for drafting an email.
  • Persistence: Persistence is evaluated separately by testing whether a malicious memory continues to hijack later tasks after k benign turns without reinjection.This probes long interaction horizons rather than only immediate cross-task transfer.
  • Resistance: Resistance tests whether later benign tasks relapse to malicious tool choice after an explicit textual correction while poisoned memory remains retrievable.The criterion focuses on recurrence after correction, not merely the initial attack.

G.2 Experimental Results with ISR

The ISR results separate successful memory injection from subsequent control-flow deviation. Across direct-write settings, injection succeeds consistently, so ASR differences reflect memory-induced deviations rather than failed memory construction.

  • ISR interpretation: Injection succeeds consistently across the main direct-write settings, making ASR differences attributable to memory-induced control-flow deviation.ISR measures whether the action-oriented adversarial policy is written into memory before auditing.
  • Results presentation: Table 8 reports ISR alongside ASR under matched controls and core evaluation settings.The clean baseline and benign-memory control are merged because both yield 0% ASR in corresponding settings.
  • Open-weight evaluation: The open-weight Qwen evaluation reuses the two-phase injection-to-auditing protocol with clean, benign-memory, and malicious-memory conditions.LlamaIndex results are not reported because the implementation does not support Qwen.

G.3.3 Results

Additional experiments show that the main qualitative MCFA pattern extends to an open-weight backbone and remains robust when decoding becomes more stochastic. Matched controls stay at 0% ASR, while malicious memory continues to induce substantial or near-saturated attacks.

  • Open-weight results: Matched controls remain at 0% ASR, while malicious memory induces near-saturated Override, M-Scope, Persistence, and Resistance attacks on Qwen.Order remains non-saturated but still substantial under strict scoring.
  • Controlled variants: The controlled variants test MCFA sensitivity to decoding temperature and injection-prompt explicitness under the representative Override setting.They retain the same tool inventory, trace-auditing procedure, and success criterion as the main evaluation.
  • Temperature sensitivity: Override ASR remains saturated or near-saturated across all three LLM backbones at temperatures 0.0, 0.5, and 1.0.The main experiments use temperature 0.0, while additional tests evaluate 0.5 and 1.0.
  • Interpretation: The results indicate that retrieved malicious memory continues steering tool selection toward the adversarial policy under more stochastic sampling.This indicates the vulnerability is not a byproduct of one deterministic decoding configuration.

I Limitations

The evaluation uses controlled auditing rather than fully deployed production agents and covers only part of the memory-architecture design space. The defense analysis is preliminary because RBMS reduces but does not eliminate attack success.

  • Scope boundary: The evaluation uses controlled auditing settings rather than fully deployed production agents, so real-world workload diversity, ambiguity, and detectability are not fully captured.Tasks are grounded in LangChain and LlamaIndex ecosystems but remain controlled.
  • Memory coverage: The study does not cover the full design space of memory architectures and memory-management strategies.Evaluated settings include single-channel memory, noisy stores, memory summarization, and RBMS-style segregation.
  • Defense limitation: RBMS reduces attack success but does not eliminate it, leaving stronger deployment-specific defenses for further evaluation.This is identified as a preliminary defense analysis.
  • Experimental setting: Experiments use synthetic tasks and safe/risky tool variants without real user data, credentials, deployment systems, or real harmful actions.Tool calls are logged for trace-level evaluation, and deployment-ready exploit code against real services is not released.
Loading 2603.15125v3…