PaperlayerView exact revision on arXiv

Source-linked AI summary

SPA: Securing Persistent LLM Agents Across Queries with Plan-First Information-Flow Control

Dylan Girrens, Guangjing Wang

arXiv:2608.27234v1cs.CRcs.SE

TL;DR

Persistent LLM agents can carry untrusted content into later queries or security-sensitive tool calls. SPA generates complete plans before tool execution and tracks information flows across persistent artifacts; under the tool_knowledge attack, attack success is 0.0% on AgentDojo and 0.2% on AgentDojo-MQ.

  • Problem

    Persistent LLM agents face untrusted data that can redirect actions, enter security-sensitive tool arguments, or persist into later queries, while existing AgentDojo tasks reset state and cannot measure delayed attacks.

    Source: §1 · pp. 1–2

  • Method

    SPA generates a complete declarative plan before tool execution, verifies confidentiality and integrity flows, and preserves security labels on stored results while revealing only semantic metadata during later planning.

    Source: §1 · p. 2

  • Results

    Under the tool_knowledge attack, IFC reduces attack success to 0.0% on single-query AgentDojo and 0.2% on AgentDojo-MQ, while lowering concrete-mode baseline utility from 53.0% to 29.0% on AgentDojo and from 62.5% to 35.3% on AgentDojo-MQ.

    Source: §1 · p. 2

  • Takeaways & Limitations

    AgentDojo-MQ makes cross-query risks measurable, exposing delayed attack paths and the security–utility tradeoffs of strong enforcement.

    Source: §9 · p. 13

    The evaluation does not exhaustively characterize cross-query attacks, and AgentDojo’s tool_knowledge attack only weakly exercises delayed memory poisoning.

    Source: §8 · p. 13

Abstract

from arXiv · show

Large language model (LLM) agents increasingly operate over untrusted webpages, documents, tools, and persistent states while exercising authority over security-sensitive resources. Existing defenses typically protect either planning or individual tool interactions, but persistent agents face a broader threat: attacker-controlled data can alter control flow, enter security-sensitive tool arguments, or compromise later queries. We present SPA, a plan-first architecture that secures planning, execution, and cross-query state reuse. SPA invokes the planner once per query to generate a complete executable plan in a declarative domain-specific language, then applies dual-lattice information-flow control to track confidentiality and integrity across explicit data flows and control dependencies. To support persistence without re-exposing untrusted payloads to the planner, SPA stores execution results as labeled artifacts and reveals only semantic metadata during later planning. We evaluate SPA on AgentDojo and AgentDojo-MQ, which is our multi-query extension for measuring secure state reuse and delayed attacks. Under the 'tool_knowledge' attack, SPA with information-flow control reduces attack success to zero on AgentDojo and 0.2% on AgentDojo-MQ. Our results show that plan-first execution combined with label-preserving persistence can substantially strengthen persistent LLM agents, while revealing an important security-utility tradeoff introduced by strict integrity enforcement.

1 Introduction

Persistent LLM agents can let untrusted content alter control flow, enter sensitive tool arguments, or become actionable in later queries, so protection must span planning, execution, and state reuse. SPA uses complete plans, information-flow control, and labeled persistent artifacts to address these risks.

1 Introduction

  • Persistent reuse creates a tension: hiding stored payloads protects planning integrity, but hiding all prior state prevents useful reuse, while fixed plans still require control of information flows.Future values may affect security-sensitive arguments or control-dependent actions, so SPA must reason about confidentiality and integrity across explicit data and implicit control dependencies before execution.Source: §1 · p. 2
  • SPA invokes the planner once per query to generate a complete executable DSL plan, then checks confidentiality and integrity across data and control dependencies before execution.The restricted DSL makes dependencies explicit; SPA rejects plans that would disclose sensitive information or let low-integrity data drive higher-integrity actions. Its optional abstract-planning mode hides installed tool metadata and binds synthesized capabilities after planning, failing closed when no compatible mapping exists [15].Source: §1 · p. 2
  • SPA stores execution results as labeled artifacts, exposing only semantic metadata to later planners and retrieving concrete values with their original labels during execution.This allows later queries to identify relevant prior results without returning stored payloads to the planner.Source: §1 · p. 2
  • AgentDojo-MQ extends AgentDojo with multi-query episodes and explicit cross-turn artifact dependencies to measure persistent-state reuse and delayed attacks.The original AgentDojo tasks reset agent state and therefore cannot measure whether adversarial information survives one query and becomes relevant to another.Source: §1 · p. 2
  • Dual-lattice IFC reduces tool_knowledge attack success to 0.0% on AgentDojo and 0.2% on AgentDojo-MQ.IFC lowers concrete-mode baseline utility from 53.0% to 29.0% on AgentDojo and from 62.5% to 35.3% on AgentDojo-MQ, mainly because integrity checks reject plans that pass low-integrity tool outputs to higher-integrity actions. Later turns retrieve required producer artifacts in 95.4% of reuse opportunities without IFC and 89.9% with IFC.Source: §1 · p. 2

2 Background and Related Work

This section defines prompt-injection threats to agents, introduces confidentiality and integrity information-flow control, and reviews defenses that constrain planning, execution, and tool use.

2.1 Prompt Injection Attacks

  • Prompt injection embeds adversarial instructions in LLM inputs, either directly in user-facing prompts or indirectly in content agents later read [11] [19] [30].The underlying issue is that instructions and data share the same natural-language channel.Source: §2.1 · p. 3
  • Prompt injection can manipulate tool control flow, pass attacker-chosen values to sensitive tools, or exploit malicious tool metadata that biases planning [7] [15] [16] [22].Data-flow attacks may preserve the user’s intended action sequence while substituting an attacker-chosen value, such as a transfer destination.Source: §2.1 · p. 3

2.2 Information-Flow Security

  • Information-flow control assigns security classes to values, restricting flows by policy; confidentiality limits disclosure, while integrity prevents low-trust data from influencing higher-trust actions.The confidentiality and integrity models follow Bell–LaPadula and Biba, respectively.Source: §2.2 · p. 3
  • FIDES [5] dynamically combines labels of information exposed to the planner into a dual confidentiality–integrity context label and checks each tool call and its arguments against policy.Calls that violate tool confidentiality or integrity policies are blocked.Source: §2.2 · p. 3
  • CaMeL [7] labels values with provenance and recipient capabilities, enabling tool-specific confidentiality and integrity policies beyond simple lattice comparisons but relying on correctly authored policies.Its capabilities record data origins and permitted recipients.Source: §2.2 · p. 3

2.3 Plan-First Defenses and Abstract Tools

  • F-secure [27] separates planner and executor and generates plan steps under an integrity lattice, but does not check argument-level integrity on every tool invocation.Its execution-trace guarantee prevents untrusted execution results from returning to the planner, although the executor may still process untrusted data during a generated step.Source: §2.3 · p. 3
  • CaMeL [7] separates a privileged planner from a quarantined LLM for untrusted outputs and mediates tool calls through capabilities and tool-specific policies.Because the privileged planner does not see execution values, malicious tool outputs cannot directly alter its previously generated plan.Source: §2.3 · p. 3
  • ACE [15] plans with abstract tools synthesized from the user request, then binds them to installed tools, preventing installed-tool metadata from directly influencing planning.Execution may fail if no installed tool matches a synthesized capability or the request lacks enough information to define it.Source: §2.3 · p. 3

3 Threat Model

SPA’s threat model covers adversarial information entering through tools or retrieved content in a persistent agent, where it can affect planning, sensitive actions, confidentiality, or later queries. The model trusts user queries and deployment configuration, and excludes attacks unrelated to information flow.

Threat scope

  • SPA models a persistent agent serving sequential queries, with attacks through tools or external content that can alter plans, influence sensitive arguments, disclose secrets, or lie dormant in state.Attacks may affect the current query or become actionable in a later query.Source: §3 · p. 4

Adversary capabilities

  • The adversary may control tool metadata, tool implementations, runtime outputs, or content later retrieved from emails, webpages, documents, databases, and other resources.Adversarial content may persist in the external environment and be encountered repeatedly during a session.Source: §3 · p. 4

Trust assumptions

  • User queries and deployment configuration are trusted; direct prompt attacks are out of scope, and deployers configure the security lattices, tool policies, and tool-identity bindings.Source: §3 · p. 4

Attacker objectives

  • The attacker targets planning integrity, execution integrity, and confidentiality, including delayed attacks in which stored adversarial information affects a later plan or consequential tool call.Persisted artifacts are subject to the same confidentiality and integrity restrictions as fresh tool outputs.Source: §3 · p. 4

Security goals

  • SPA seeks planner isolation, policy-compliant execution integrity and confidentiality, and cross-query safety through trusted metadata in planning and label-restoring retrieval in execution.It does not defend against model-level jailbreaks or independent alignment failures, or side channels below the trusted computing base.Source: §3 · p. 4

4 System Design

SPA generates and verifies a complete plan before executing it with a deterministic interpreter, separating planning from execution-time data. It persists results as labeled artifacts while exposing only semantic metadata to later planning.

4.1 System Overview

  • For each query, SPA generates a complete plan before tool execution, verifies its information flows, and executes the accepted plan exactly with a deterministic interpreter.Unsafe plans are rejected, and execution results are committed with verified labels and planner-visible metadata; any procedure failure terminates the query, and the planner is invoked only once.Source: §4.1 · pp. 4–5

4.2 Trusted Planning

  • Abstract planning keeps installed-tool metadata from the planner by synthesizing query-specific tool capabilities, then mapping planned abstract tools to compatible installed implementations.Generated input/output transformations run in a restricted interpreter; if any required tool cannot be mapped, execution does not proceed.Source: Figure 2 · p. 5, §4.2 · p. 6

4.3 Isolated Execution

  • Execution follows the verified plan through per-tool backends, JSON Schema validation, and a quarantined LLM that processes untrusted text without changing its inherited security labels.An optional ephemeral container has no network access, a read-only filesystem, an unprivileged user, and a wall-clock timeout; the quarantined LLM has no tool access and must return schema-conforming output.Source: §4.3 · p. 6

4.4 Persistent Context

  • Persistent artifacts separate concrete values, security labels, and planner-visible metadata, allowing later queries to identify relevant results without exposing stored payloads to the planner.Metadata includes the output schema, intent and output_semantics annotations, and producing query identifier; values re-enter execution only through explicit retrieval, which restores their original labels.Source: §4.4 · pp. 6–7

5 Domain-Specific Planning Language

SPA’s DSL makes plans statically checkable and reliably generable while expressing common workflows and supporting persistent artifact reuse. Its restricted syntax and step forms make data and control dependencies explicit for information-flow analysis.

5.1 Design Goals

  • JSON plans support static syntax and information-flow checking, reliable generation, common agent workflows, and semantic metadata for recognizing persisted values across queries.The design favors a small, uniform set of step types; JSON is compatible with constrained decoding and supports per-step annotations and reuse metadata.Source: §5.1 · p. 7

5.2 Syntax

  • Plans are ordered step lists whose restricted expressions, explicit artifact retrieval, and fixed predicate operators expose data and control dependencies for static analysis.Expressions permit only references, projections, literals, and constructed JSON objects or arrays; compound logic uses nested conditionals rather than compound predicates.Source: §5.2 · p. 7

5.3 Step Forms

  • Retrieve loads persisted artifacts, tool steps alone invoke external capabilities and cause side effects, and quarantined-LLM steps convert unstructured inputs into schema-conforming structured values.Quarantined-LLM content is kept isolated from the planner.Source: §5.3 · p. 8
  • Display returns information to users, conditions execute exactly one branch, and loops execute a body once per collection element.Tool and quarantined-LLM steps require planner-written intent and output-semantics annotations based on trusted inputs.Source: §5.3 · p. 8

5.4 Well-Formed Plans and Execution

  • Validation checks plan structure, bindings, scopes, artifact references, and predicates, while a fail-closed interpreter terminates on projection errors, schema violations, or tool failures.Tool-argument schema conformance is checked at invocation time; explicit dependencies and dedicated effect steps allow static flow analysis before execution.Source: §5.4 · p. 9

6 Information Flow Model

SPA assigns values separate confidentiality and integrity labels, propagates them through data and control dependencies, and checks plans against endpoint policies before execution. Its verifier accounts for tool outputs, persistent artifacts, control flow, and user-visible display.

6 Information Flow Model

  • Each value receives a product label: dependencies raise confidentiality and lower integrity, so derived values are at least as sensitive and no more trusted than their inputs.The deployment defines finite confidentiality and integrity lattices, which need not be totally ordered.Source: §6 · p. 9

6.1 Flow Rule and Endpoint Policies

  • A flow is permitted only when the destination protects confidentiality at least as strongly as the source and requires no more integrity than the source provides.Tool policies specify maximum input confidentiality, minimum input integrity, output baselines, and a separate policy for the display channel.Source: §6.1 · p. 9
  • Query labels initialize the program counter, while persistent artifacts restore their stored labels when retrieved into a later query.Literals in the generated plan inherit the query label; tool outputs combine argument labels with the tool’s output baseline.Source: §6.1 · p. 9

6.2 Label Propagation

  • Verification combines data-dependency labels with the current program-counter label; tool calls must satisfy endpoint policy, while tool results also inherit the tool’s output baseline.A quarantined LLM’s output inherits its inputs’ combined label, preventing it from laundering low-integrity content into a more trusted value.Source: §6.2 · pp. 9–10
  • Conditional branches and loop bodies incorporate guard or collection labels into the program counter, so actions inherit restrictions from the data controlling their execution.Both conditional branches use the same updated program counter because the verifier cannot statically determine which branch will run.Source: §6.2 · p. 10
  • Display items that violate the sink policy are selectively redacted, while any plan violating a tool policy fails closed and never reaches the executor.The verifier makes one static pass and returns either a labeled plan or a violation.Source: §6.2 · p. 10

7 Evaluation

SPA is evaluated on AgentDojo and the multi-query AgentDojo-MQ extension for task utility, resistance to prompt injection, and persistent-artifact reuse without exposing payloads to the planner.

7.1 Construction of AgentDojo-MQ

  • AgentDojo-MQ converts the 97 AgentDojo tasks into multi-turn episodes with artifact produce/dependency annotations, preserving tools and checks and applying injections once at episode start.A task-specific dependency graph and deterministic segmenter create two to five turns, with artifact IDs marking what later turns should reuse.Source: §7.1 · p. 11

7.2 Experimental Setup

  • The evaluation uses four planner/IFC configurations, plus an open-memory control on AgentDojo-MQ, where persisted values are exposed to later planners.The benchmark has 97 tasks across four suites; tool_knowledge gives the adversary the target task’s ground-truth tool sequence, producing 949 attack runs per configuration.Source: §7.2 · p. 11

7.3 Metrics

  • The evaluation measures task-check utility, attacker-goal ASR, pipeline execution status, and AgentDojo-MQ reuse of artifacts from earlier producer turns.Soft-hit reuse averages retrieval over scorable turns, whereas strict reuse counts missing producer artifacts as misses; macro averages are attack-run-weighted except baseline utility comparisons across settings.Source: §7.3 · pp. 11–12

7.4 Single-Query AgentDojo Results

  • On single-query AgentDojo, IFC reduces Concrete ASR from four banking successes to 0% across 949 attack runs, while baseline utility falls from 53.0% to 29.0%.About 53% of Concrete IFC trials stop after an integrity rejection; abstract configurations are chiefly limited by tool-mapping failures, with about 9–12% reaching execution.Source: Table 1 · p. 12, §7.4 · p. 12

7.5 Multi-Query AgentDojo-MQ Results

  • On AgentDojo-MQ, Concrete IFC has 0.2% ASR but 35.3% baseline utility versus 62.5% for metadata-only Concrete No IFC.Across configurations, ASR remains low; the open-memory control reaches 0.6% ASR and 65.9% baseline utility, while exposing persisted values raises utility under attack from 60.0% to 66.8%.Source: Table 2 · p. 12, §7.5 · p. 12
  • Open memory raises baseline utility from 62.5% to 65.9% versus metadata-only Concrete No IFC, but its effects vary by suite and tool_knowledge weakly probes delayed injection.Banking baseline utility rises from 56.2% to 81.3%, while travel falls from 70.0% to 50.0% on 20 tasks; visible persisted payloads do not produce a large delayed-injection ASR under this attack.Source: §7.5 · p. 12
  • Concrete No IFC achieves 95.4% soft-hit reuse and 80.1% strict reuse on AgentDojo-MQ, with 16.8% of reuse opportunities unscorable.Concrete IFC reaches 89.9% soft-hit reuse; abstract configurations have 81–87% unscorable rates and only 12.5–14.7% strict reuse. Soft-hit reuse excludes failed producer turns.Source: Table 3 · p. 13, §7.5 · p. 13
  • Relative to single-query AgentDojo, AgentDojo-MQ Concrete No IFC baseline utility is 8.2 percentage points higher, and Concrete IFC is 5.2 points higher.These comparisons use baseline-run-weighted macros, compare task formulations rather than isolating persistence, and soft-hit reuse can overstate performance by excluding failed producer turns.Source: §7.5 · p. 13

8 Limitations and Future Work

SPA establishes a conservative security boundary for persistent agents, but its evaluation, integrity enforcement, trusted policies, and multi-query benchmark construction leave important limitations and future-work directions.

Limitations and Future Work

  • AgentDojo’s tool_knowledge attack weakly exercises delayed memory poisoning, motivating persistence-specific tests for delayed control-flow injection and later sensitive tool arguments.The current evaluation primarily targets injected tool outputs; future attacks should test whether attacker-controlled values enter security-sensitive arguments in later queries.Source: §8 · p. 13
  • Strict Biba-style integrity enforcement can reject legitimate workflows that reuse externally controlled data in higher-integrity actions, reducing concrete-mode utility.Potential remedies include field-level labels, argument-specific integrity requirements, and scoped endorsement after an explicit check.Source: §8 · p. 13
  • SPA trusts its lattices, per-tool policies, and tool-to-implementation bindings, so policy errors can reject safe workflows or admit unsafe ones.Future work should automate policy inference, audit configurations, and measure robustness to policy misclassification.Source: §8 · p. 13
  • AgentDojo-MQ manually converts 97 AgentDojo tasks into dependency-graph-based multi-query episodes, limiting coverage of natural long-running interactions and model-dependent planning.Broader evaluation should use independently collected multi-session workloads, additional planner models, and stronger validation of dependency annotations.Source: §8 · p. 13

9 Conclusion

SPA treats prompt injection in persistent LLM agents as a cross-query information-flow problem. It combines executable plans, label-preserving persistent artifacts, and AgentDojo-MQ to make cross-query behavior enforceable and its risks measurable.

9 Conclusion

  • SPA generates complete executable plans that expose future data and control dependencies before any tool runs.This makes cross-query agent behavior statically enforceable and secure.Source: §9 · p. 13
  • Label-preserving artifacts carry confidentiality and integrity constraints across query boundaries without re-exposing payloads to the planner.Source: §9 · p. 13
  • AgentDojo-MQ makes cross-query risks measurable, including delayed attack paths and security–utility tradeoffs of strong enforcement.Source: §9 · p. 13

Ethical Considerations

The study evaluates prompt-injection defenses in a controlled benchmark without human subjects, private user data, or production-system interactions. It aims to reduce unauthorized tool use and disclosure, while noting safeguards needed for deployment.

Ethical Considerations

  • Experiments use the controlled AgentDojo benchmark and involve no human subjects, private user data, or interaction with production accounts or external systems.Source: Ethical Considerations · p. 14
  • The work is designed to reduce, rather than enable, unauthorized tool use and disclosure.Source: Ethical Considerations · p. 14
  • Deployments should use appropriate access controls, least-privilege tool permissions, and safeguards for storage; the authors report no previously undisclosed vulnerability requiring coordinated disclosure.Source: Ethical Considerations · p. 14

Open Science

The paper releases an anonymized archive containing artifacts for evaluating SPA and reproducing reported result summaries.

Open Science

  • The archive includes SPA’s plan-first agent, DSL validator and executor, dual-lattice IFC analysis, persistent context store, planning modes, and CLI.The implementation is organized under agent/, dsl/, and cli/.Source: Open Science · p. 14
  • The archive provides an AgentDojo integration with a pipeline adapter, suite IFC policies, and runners used for Section 7.4.Source: Open Science · p. 14
  • AgentDojo-MQ artifacts include multi-query turn specifications, artifact dependency graphs, a session-scoped adapter, reuse scoring, and runners for Sections 7.1–7.5.They also include segmentation and phrasing pipelines.Source: Open Science · p. 14
  • The release includes documentation, setup instructions, architecture notes, unit tests, and scripts to reproduce reported result summaries.Source: Open Science · p. 14

A Prompts

Appendix A presents the prompt templates and specifications for SPA’s planning and quarantined-LLM stages, including how plans, responses, tool suites and mappings are generated and constrained. It also covers how AgentDojo-MQ tasks are phrased as multi-turn episodes while preserving turn dependencies without adding undisclosed information.

A.1 DSL Planner

  • The planner prompt combines the DSL contract with optional environment context, reference plans, prior-query records, artifact metadata, the tool catalog, and the user query.The catalog is the installed tool set under concrete planning and a synthesized abstract suite under abstract planning.Source: §A.1 · pp. 15–16
  • The DSL planner emits one executable plan as a single JSON object with an ordered `steps` array and a fixed set of step types.The supported types are `retrieve`, `tool`, `q-llm`, `display`, `condition`, and `loop`.Source: §A.1 · p. 16
  • Its specification defines typed variable references, projection paths, and scope and definition-before-use rules for plan steps.Projected references extract a field or index when a downstream consumer requires a scalar or nested value.Source: §A.1 · p. 16
  • Plans use registered tools for deterministic behavior and `q-llm` steps for tasks such as extraction, summarization, or classification.A `q-llm` step has no tools or plan memory and returns an object constrained by its output schema; its prompt must describe the objective, inputs, and schema fields.Source: §A.1 · p. 16, §A.2 · p. 17
  • For persistence, the planner receives artifact metadata rather than values, uses matching artifacts by retrieving them, and avoids recreating existing work when suitable.Metadata includes descriptive fields and schema; artifact values are loaded at runtime by `retrieve`.Source: §A.1 · pp. 16–17

A.2 Quarantined LLM

  • The quarantined-LLM prompt presents a step objective and inputs as JSON, while the API constrains the reply to the step’s output schema.Its fixed instructions frame it as a structured-output helper receiving `prompt` and `inputs`.Source: §A.2 · p. 17
  • The quarantined LLM prompt asks the model to interpret supplied inputs in context and return exactly one schema-matching JSON object without tools or commentary.If the objective cannot be completed from the supplied inputs, it directs the model to use an explicit failure representation when available.Source: §A.2 · p. 17

A.3 Abstract Tool Synthesis

  • Abstract tool synthesis creates a minimal, reusable suite from the query before planning, prioritizing environment capabilities over tools tailored to one execution plan.Tools should cover the required environment, including small mixes of domains, while leaving natural-language reasoning and other LLM-capable work to the model.Source: §A.3 · p. 18
  • Synthesized schemas keep convenience filters and nonessential output details optional, and include tools for current or live data when needed.The prompt cautions against excessive filtering and over-specified outputs, favoring a usable primary payload and only truly necessary required arguments.Source: §A.3 · p. 18

A.4 Abstract-to-Concrete Tool Mapper

  • The mapper rejects tools with divergent purposes or mappings that require fabricated facts, while allowing compatible schema reshaping and omission of unsupported optional fields.It emits executable input and output mappings or a failure; missing required abstract output that cannot be honestly derived also requires failure.Source: §A.4 · p. 18
  • Abstract-to-concrete mappings pass through schema validation at each stage, and execute in a restricted interpreter that rejects unlisted global names.Plan arguments are checked against the abstract input schema; mapped arguments and tool returns are checked against their respective concrete schemas before output mapping.Source: §A.4 · pp. 18–19

A.5 AgentDojo-MQ Episode Phrasing

  • AgentDojo-MQ turns each single-message task into ordered user prompts so later turns can reuse earlier results rather than receiving everything in one message.A deterministic TurnBrief packages the task into turns derived from a dependency graph, including each turn’s goal, produced intermediates, and dependencies.Source: §A.5 · p. 19
  • Each episode prompt should sound like a natural user utterance, cover only its turn’s goal, and avoid inventing values or knowledge absent from the original prompt.The prompt writer may repeat original literals but must not supply values the agent needs to discover or derive from prior turns.Source: §A.5 · p. 19

B Additional Figures

The appendix illustrates how SPA preserves security labels across queries and how AgentDojo-MQ turns single-query tasks into multi-turn episodes for evaluating persistent agents.

B Additional Figures

  • Persisted artifacts expose schema and planner-authored intent and output_semantics, but seal payloads; explicit retrieval restores original labels so IFC can block unsafe cross-query flows.Figure 6 illustrates a malicious tool output persisting into a later query and being blocked at an unauthorized sink.Source: B Additional Figures · p. 19, Figure 6 · p. 20
  • AgentDojo-MQ converts all 97 AgentDojo tasks into multi-turn episodes while preserving their tools, environments, injection setup, and final utility and security checks.Standard AgentDojo treats tasks as independent single-query trials, so it does not test cross-query persistence or delayed consumption of earlier outputs.Source: B Additional Figures · p. 19
  • The construction pipeline classifies task dependencies, builds artifact dependency graphs, deterministically segments turns with produces and depends_on annotations, and emits natural utterances as executable MQ tasks.The non-exclusive dependency classes capture how intermediate artifacts, choices, and predicates constrain later actions and where tasks can be divided without changing semantics.Source: B Additional Figures · p. 19, Figure 7 · p. 20, Figure 8 · p. 20
  • AgentDojo-MQ evaluates utility and security once after the final turn; its workspace example separates event selection from a later invitee-email request that reuses the earlier result through retrieve.This contrasts with the conventional single-prompt trial and avoids redundantly invoking tools to reconstruct the earlier result.Source: B Additional Figures · p. 19, Figure 9 · p. 21, Figure 10 · p. 21

C Additional Charts

The additional charts show higher utility for concrete than abstract planning across suites, low measured attack success, and tradeoffs from integrity enforcement and persistent-state reuse. They also show that open-memory effects vary by suite and that MQ–SQ differences cannot be attributed to persistence alone.

Additional Charts

  • IFC lowers concrete utility by 25.0 percentage points on workspace, 28.6 on slack, 35.0 on travel, and 6.25 on banking.Coarse variable-level integrity and confidentiality labels block low-integrity values from trusted inputs, including legitimate data-dependent actions because SPA has no endorsement mechanism; abstract runs usually fail during mapping, while concrete runs can fail from incomplete or incorrect programs. Banking’s 31.25% abstract baseline utility comes from five tasks whose checks pass initially despite no completed execution.Source: Figure 11 · p. 22, C Additional Charts · p. 22
  • Four of 949 Concrete No IFC attack trials succeed, all on banking task 12; IFC blocks all four, which cause three unauthorized transfers and one unauthorized password change.Plan-first isolation is a first barrier, not a guarantee: injected content interpreted by a quarantined LLM reaches trusted-input tools when IFC is disabled. The tool_knowledge injections were not designed to target persistence-specific attacks, so the low measured ASR does not establish exhaustive coverage of those attacks.Source: Figure 12 · p. 22, C Additional Charts · p. 22
  • Across 97 baseline tasks and 949 attack trials per configuration, concrete planning has higher utility in every suite, aggregate attack success stays below 1%, and abstract mapping fails in approximately 75–77% of trials.Abstract-mode low ASR must be interpreted alongside its low execution coverage.Source: C Additional Charts · p. 22, Figure 13 · p. 23, Figure 14 · p. 23
  • Exposing stored values to the planner raises banking baseline utility from 56.25% to 81.25% but lowers travel from 70.0% to 50.0%.Concrete No IFC + Values is an open-memory control; exposing values therefore does not uniformly improve planning. By default, SPA retains labeled artifacts across queries but exposes only artifact metadata to subsequent planners.Source: C Additional Charts · p. 22, Figure 14 · p. 23
  • Concrete No IFC achieves 95.4% episode-macro soft reuse on scorable episodes and 80.1% strict reuse, while 16.8% of opportunities are unscorable because no producer artifact exists.Suite-level soft reuse ranges from 90.0% to 100%; soft reuse measures retrieval conditional on an available artifact, while strict reuse captures producer failures end to end. Abstract-mode rates are less informative because about three quarters of trials never execute and producers rarely persist a candidate artifact.Source: C Additional Charts · p. 22, Figure 15 · p. 24, Figure 16 · p. 24
  • Concrete utility is higher in MQ than SQ by +8.2 / +5.2 baseline, while abstract utility barely moves.Figure 17 is not a persistence ablation: MQ also changes prompt scope, retained conversation and environment state, and episode-level scoring, so the difference cannot be attributed to persistence alone.Source: C Additional Charts · p. 22, Figure 17 · p. 24

Related papers

Loading 2608.27234v1…