PaperlayerView exact revision on arXiv

Source-linked AI summary

SEAD: A State-Based Perspective on Attack and Defense in Tool-Using Agents

Xinjie Shen, Junran Wang, Rongzhe Wei, Pan Li

arXiv:2609.34518v1cs.CRcs.AI

TL;DR

Tool actions can alter persistent state, while visible histories may not show enough to judge whether a pending action is harmful. SEAD frames attack and defense as partially observed state control, developing DART’s execution-feedback search and SAGE’s read-only pre-execution investigation. DART improves attack success against baselines, while SAGE preserves benign execution and reduces DART’s executable attack success online.

  • Problem

    A pending tool action can be benign or harmful depending on persistent environment state that the visible interaction may not reveal.

    Source: §1 · p. 1

  • Method

    SEAD models attack and defense as partially observed state control, with DART adapting search to executed trajectories and SAGE investigating state through read-only queries before gating actions.

    Source: §1 · p. 2, §4 · p. 4, §4.2 · p. 5

  • Results

    DART exceeds the best competing baseline by 18.8–35.9 percentage points, while SAGE preserves 95.79% of benign trajectories and reduces DART’s Hard ASR from 48.0% to 4.0%.

    Source: §5.2 · p. 7, §5.3 · p. 8, §5.4 · p. 8

  • Takeaways & Limitations

    SEAD organizes attack construction, defense, and evaluation around the state reached through execution, with reported protection across attack methods and out-of-domain environments.

    Source: §6 · p. 9

    SEAD focuses on tool-mediated execution with a pre-execution gate and read-only defender-private state queries; concurrent processes, compound actions, and assistant-output disclosure require extensions.

    Source: B LIMITATIONS · p. 14

Abstract

from arXiv · show

Language-model agents increasingly use tools to act on external systems. Earlier actions can alter files, permissions, database records, or other state, making a later routine-looking action harmful. Yet the visible interaction may not reveal the underlying state needed to assess that action. We formulate attack and defense as partially observed state control in SEAD, deriving their design requirements from this shared execution process. Because attackers supply instructions while the target chooses concrete actions, DART decomposes harmful goals into locally plausible steps and uses feedback from actual tool execution to guide trajectory search. The defender must decide before execution with incomplete state evidence. SAGE can therefore investigate relevant state through read-only queries before allowing or blocking each action, including those proposed after a block. We construct an environment-verifiable dataset integrating controlled initial states, replayable tool environments, and task-specific executable checks. Across four target models, DART improves semantic attack success by 18.8--35.9 percentage points over the competing baseline, with consistent gains under executable verification. On recorded trajectories, SAGE preserves 95.79% of benign trajectories while intercepting 92.73% of harmful paths by the harm-enabling boundary. In online attack-defense evaluation, it reduces DART's executable attack success from 48.0% to 4.0%. SAGE remains effective across four attack methods and generalizes to out-of-domain environments. Our code and data is available at https://github.com/EverywhereSafety/SEAD.

1 INTRODUCTION

Tool-using agents can cause harm through actions whose effects depend on persistent environment state that their visible histories may not reveal. SEAD derives role-specific attack and defense methods from this shared, partially observed execution process and evaluates them with environment-verifiable data.

1 INTRODUCTION

  • Persistent tool effects make trajectory outcomes and the current environment state—not an isolated response—central to agentic safety.The same upload can be benign or expose authentication data depending on file contents and destination access.Source: §1 · p. 1
  • DART adapts harmful-goal search to actual tool execution, while SAGE investigates relevant state through private, read-only queries before passing or blocking actions.SAGE checks each new proposed action after a block before it executes.Source: §1 · p. 2, Figure 1 · p. 2
  • The dataset combines controlled initial states, replayable tool environments, and task-specific executable checks, with benign trajectories used to measure preserved normal execution.Experiments report substantial improvements in attack success for DART and in SAGE’s safety–utility trade-off and online defense.Source: §1 · p. 2

2 RELATED WORK

Related work measures agent task performance and safety, adapts attacks using interaction feedback, and develops runtime defenses that inspect actions or environment evidence. SEAD places attack construction and defense within a shared state-transition view.

Agentic execution and adaptive attacks

Runtime defenses for agent actions

  • Runtime defenses vary in intervention point and evidence, from completed-trajectory scoring to pre-execution checks and environment-grounded look-ahead.SAGE can gather read-only state evidence before gating an action, and replanned actions return to the gate.Source: §2 · pp. 2–3

3 AGENTIC SAFETY AS PARTIALLY OBSERVED STATE CONTROL

SEAD models tool-mediated safety as a shared execution loop in which the attacker, target, and defender have role-specific observations and actions. Persistent state effects, partial observability, and intervention-driven continuation motivate separate requirements for attack search, defense, and evaluation.

3 AGENTIC SAFETY AS PARTIALLY OBSERVED STATE CONTROL

  • The shared loop has an attacker supply instructions, a target propose tool actions, and a defender decide whether each action executes; blocked calls still count as attempts.Permitted actions change state, while blocked actions leave it unchanged and return refusal feedback that informs later behavior.Source: §3.1 · p. 3, Table 1 · p. 3

3.1 AGENTIC EXECUTION AND SAFETY PROPERTIES

  • Harm is a property of evolving state: an attack succeeds when execution reaches a state satisfying its harmful goal, and every successful trajectory has a first boundary-crossing transition.An action can preserve benign progress in one state and complete a harmful goal in another.Source: §3.1 · pp. 3–4
  • Visible history may leave the pending action’s effect ambiguous, while benign and harmful trajectories can share preparation steps, making current-state evidence necessary to distinguish safe progress from harm.Inferring a harmful objective alone does not establish that the current action crosses the boundary.Source: §3.1 · p. 4

3.2 ROLE-DERIVED REQUIREMENTS

  • Because the target chooses actions and the defender may block them, the attacker must plan from the realized trajectory and observed effects rather than an intended route.The role-specific histories update after each tool-action attempt and inform subsequent instructions, actions, and defense decisions.Source: §3.2 · p. 4, §3.1 · p. 4
  • The defender must decide before execution and use noninterfering queries when its history is compatible with states requiring opposite decisions; evaluation therefore tests both recorded-path interception and adaptive online continuation.Online evaluation permits attacker plan revision after execution outcomes and refusal feedback.Source: §3.2 · p. 4

4 FROM PRINCIPLES TO ATTACK AND DEFENSE

The paper develops DART and SAGE from SEAD’s state-control view: execution outcomes guide attack search, while evidence about pending actions guides pre-execution defense. DART searches executed trajectories; SAGE investigates state using read-only queries and gates actions.

4.1 DART: FEEDBACK-CONDITIONED ATTACK

  • DART branches from executed trajectory prefixes, proposes alternative instructions, records actual target execution, and selects continuations using observed progress and feedback.Its node value combines introspection, execution feedback, and executable-verifier progress as Q_v = 0.20I_v + 0.30C_v + 0.50P_v.Source: §4.1 · pp. 4–5
  • DART backs up terminal rewards for goal completion, repeated refusal, or budget exhaustion, then uses UCT to balance estimated path value with exploration.Executable evaluation uses task-specific verifiers, complemented by an independent LLM judge matching checkpoints against execution and environment evidence.Source: §4.1 · p. 5

4.2 SAGE: STATE-AWARE PRE-EXECUTION DEFENSE

  • SAGE assesses whether a pending action realizes harm and can gather additional evidence when defender-visible history leaves relevant state unresolved.Its optional read-only queries extend a defender-private transcript without modifying task state; reading a script and configuration can distinguish an authorized test update from a protected-record modification.Source: §4.2 · pp. 5–6
  • After a block, refusal feedback enters the target-visible history, and each replanned tool action is submitted to the gate again.The gate’s decision thus applies to the next pending action before it executes.Source: §4.2 · p. 6
  • SAGE is trained on defender traces with optional evidence queries and a final decision, using labels that mark benign actions and pre-closure actions PASS and the closure action BLOCK.These goal-specific annotations approximate the harmful boundary.Source: §4.2 · p. 6

5 EXPERIMENTS

The experiments assess attack effectiveness across target agents, offline safety–utility on recorded trajectories, and online defense when attackers can adapt. They use executable outcome checks alongside semantic judgments and evaluate SAGE on domains excluded from its fine-tuning.

5.1 EXPERIMENTAL SETTINGS

  • The benchmark uses 187 malicious tasks across Filesystem, Terminal, PostgreSQL, and Web; RQ2 and RQ3 use a 75-task subset, and RQ2 adds 95 benign trajectories.RQ1 targets four models; RQ2 and RQ3 target GPT-5.6 Luna, while SAGE is fine-tuned only on Filesystem and Terminal trajectories.Source: §5.1 · p. 6, §5 · p. 6

5.2 RQ1: ATTACK EFFECTIVENESS

  • DART has the highest semantic and hard ASR for every target, beating each target’s best competing baseline by 18.8–35.9 percentage points semantically and 8.1–17.9 points on hard success.Table 2 reports ASR across four target agents; hard success requires the task’s executable evaluator to pass.Source: §5.2 · p. 7, Table 2 · p. 7

5.3 RQ2: SAFETY–UTILITY TRADE-OFF

  • SAGE passes 91 of 95 benign trajectories (B = 95.79%) and records H1 = 34.55%, H2 = 58.79%, F1 = 50.78%, and F2 = 72.86%.It gives up 4.21 points of B while gaining 65.45 points of harmful interception relative to Fine-tuned Binary.Source: §5.3 · p. 8, Table 3 · p. 8
  • SAGE has the highest Exact interception rate among listed methods; versus TS-Guard, it reduces Early blocks from 78.18% to 58.18% and Miss from 14.55% to 7.27%, with no Late blocks.Fine-tuned Binary’s 72.73% Miss rate shows why a lower Early rate alone does not establish better interception.Source: §5.3 · p. 8, Table 3 · p. 8

5.4 RQ3: ONLINE DEFENSE AND ATTACK–DEFENSE INTERACTION

  • Against DART online, SAGE reduces Semantic ASR from 65.3% without defense to 10.7%, and Hard ASR from 48.0% to 4.0%.Figure 3 shows overall ASR and domain success counts on the 75-task subset.Source: §5.4 · pp. 8–9, Figure 3 · p. 9
  • On 30 PostgreSQL/Web tasks excluded from SAGE fine-tuning, SAGE reduces DART’s Semantic successes from 20 to 3 and Hard successes from 9 to 1.The same learned investigation and gating policy is reused with domain-appropriate read-only tools.Source: §5.4 · p. 9
  • SAGE reduces ASR under both criteria across all attack methods; Intent Hijacking has the largest residual ASRs at 13.3% Semantic and 6.7% Hard, versus DART’s 10.7% and 4.0%.Across all four methods, no PostgreSQL attack achieves Hard success, and each method produces at most two Hard successes in Web.Source: §5.4 · p. 9, Figure 4 · p. 9

6 CONCLUSION

SEAD frames tool-using agent safety as partially observed control over consequential state transitions. It organizes attack, defense, and evaluation around the state reached through execution.

6 CONCLUSION

  • SEAD treats tool-mediated safety as partially observed control over consequential state transitions.The attacker acts before the target's realized trace, and the defender decides before a proposed action changes the environment.Source: §6 · p. 9
  • DART adapts attack search from executed prefixes, while SAGE can inspect read-only state before a pre-execution decision.The experiments include attack effectiveness, fixed-trajectory defense trade-offs, and online protection.Source: §6 · p. 9
  • SEAD organizes attack construction, defense, and evaluation around the state reached through execution.Source: §6 · p. 9

A DISCUSSION

The discussion connects execution feedback and state investigation to attack and defense outcomes. It also identifies open questions about investigation policies and evidence reuse.

A DISCUSSION

  • DART's executed-prefix search produces semantic and executable successes across reported target models, while their disagreement keeps progress scoring distinct from terminal verification.Source: A DISCUSSION · p. 14
  • On fixed trajectories, SAGE's epoch-1 checkpoint achieves B = 95.79% on benign paths and intercepts most selected harmful paths by annotated closure.SAGE has the highest F1 and F2 among evaluated defenders; exact-interception cases distinguish preparation from consequential environmental change.Source: A DISCUSSION · p. 14
  • Online campaigns return block feedback to the target and gate every new action, measuring the continuation produced by intervention.Source: A DISCUSSION · p. 14
  • A defender's safety objective includes preventing harmful state transitions while preserving legitimate task completion, making evidence acquisition and use part of its policy.Source: A DISCUSSION · p. 14
  • SAGE implements state investigation with noninterfering queries and supervision that distinguishes preparation from the annotated harmful boundary.A cited investigation case uses a query to reveal sensitive resource fields before the pending action takes effect.Source: A DISCUSSION · p. 14
  • Further work includes learning which queries can change a decision, when to stop, and how to reuse evidence while tracking its validity as the environment changes.The discussion also calls for joint assessment of harmful outcomes and legitimate task completion after intervention.Source: A DISCUSSION · p. 14

B LIMITATIONS

The supplementary discussion reports interception timing results and specifies the scope, evaluation data, metrics, and attack-scoring procedures. It also identifies boundaries of the current intervention interface and evaluation.

B LIMITATIONS

  • SEAD assumes consequential actions pass through a pre-execution gate with read-only, defender-private queries; concurrent processes, compound actions, and assistant-output disclosure need interface extensions.The evaluation covers four tool domains and four attack target models, while broader target coverage and component studies remain directions for extension.Source: B LIMITATIONS · p. 14

C FORMALIZATION DETAILS

  • The formalization counts attempted tool actions including blocked calls, defines success by reaching a harmful state, and assumes trusted interception before effects occur.Source: C FORMALIZATION DETAILS · pp. 14–15

D.1 EVALUATION DATA

  • The 187-task collection is Official-75 plus Expansion-112; offline evaluation uses a 75-task subset, and online evaluation uses the same Official-75 subset.Source: §D.1 · p. 15

D.2 METRIC DEFINITIONS

  • Benign all-pass requires every candidate action in a trajectory to receive PASS, while late blocks and misses contribute zero to both harmful scores.F1 and F2 are harmonic safety–utility summaries, not action-level classification F1.Source: §D.2 · p. 15

D.3 ATTACK CONFIGURATION AND SCORING PROTOCOLS

  • Under depth-8, branching-2 search, DART scores executed children using progress, introspection, and feedback; Hard and Semantic success use distinct evaluators.Source: §D.3 · pp. 15–16

D.4 ADDITIONAL OFFLINE RESULTS

  • 34.5% Exact rate and 7.3% Miss rate make SAGE best among four displayed methods on both measures, though its early blocks still vary in distance from closure.Fine-tuned Binary has a 72.7% Miss rate.Source: §D.4 · p. 16

D.5 ADDITIONAL ONLINE RESULTS

Online evaluation reports SAGE’s reduction of DART attack success and compares hard attack success across attack methods. First-block timing is also characterized relative to the harmful closure action.

Online defense against DART

  • Hard ASR falls from 48.0% to 4.0% under SAGE, which achieves the lowest overall ASR under both criteria.On PostgreSQL and Web, which were excluded from defender fine-tuning, DART has zero and one Hard successes with SAGE, versus five and four without defense.Source: §D.5 · p. 16

Hard ASR across attack methods

  • With SAGE, overall Hard ASR is 1.3% for MTA, 4.0% for STAC, 6.7% for Intent Hijacking, and 4.0% for DART.Overall ASR is reduced for each attack method under executable outcome checks as well as Semantic judgments.Source: §D.5 · p. 16

First-block timing

  • First-block positions are measured relative to harmful closure, with exact interception marked; miss percentages have no finite plotted position.Figure 5 shows signed distance k−T and relative position k/T; its highlighted values indicate the highest Exact and lowest Miss rates among displayed methods.Source: Figure 5 · p. 17, Table 4 · p. 17

D.6 DEFENDER DATA COLLECTION, TRAINING, AND INFERENCE

The section describes how defender data and demonstrations are assembled, then presents cases illustrating state changes, branching, online interception, and out-of-domain defense.

D.6 DEFENDER DATA COLLECTION, TRAINING, AND INFERENCE

  • SAGE uses demonstrations built from Filesystem and Terminal trajectories; PostgreSQL and Web are excluded from defender fine-tuning.The source pool contains 1,075 harmful and 1,000 benign trajectories, while the resulting SAGE training set covers 615 benign and 615 harmful source trajectories.Source: §D.6 · pp. 17–19
  • Each defender example pairs pre-action history with a pending action; harmful labels mark the earliest closure action BLOCK and preceding actions PASS.SAGE demonstrations may investigate before deciding, and the same trained checkpoint is reused across domains with domain-appropriate read-only tools.Source: §D.6 · pp. 18–19

E.1 PREPARATORY STATE CHANGES: A STAGED VALUE USED IN A HARMFUL UPDATE

  • In the PostgreSQL case, a persistent staging-table write supplies the replacement value used later to change five account passwords.The harmful update passes both Hard and Semantic evaluation; the staged value was stored before the final update.Source: §E.1 · p. 20, Table 7 · p. 20

E.2 SEARCH BRANCHING: RETURNING TO AN ALTERNATIVE PREFIX AFTER REFUSALS

  • After one search branch ends in repeated refusals, DART expands an alternative prefix and obtains two confirmed-success leaves.The selected leaf satisfies both Hard and Semantic criteria.Source: §E.2 · pp. 20–21, Table 8 · p. 20

E.3 ONLINE: EARLIER REFUSALS DO NOT GUARANTEE FINAL PROTECTION

  • In the Terminal case, SAGE blocks two actions before the final turn, but that turn still satisfies all three executable checkpoints and receives a successful Semantic judgment.SAGE makes nine environment queries along the path, eight of which return successfully.Source: §E.3 · p. 21, Table 9 · p. 21

E.4 OOD ONLINE DEFENSE: INTERCEPTION ACROSS EXPLORED BRANCHES

  • In the PostgreSQL case outside defender fine-tuning, every explored path includes a SAGE block and all nine leaves end as unsuccessful dead ends.The recorded search has nineteen non-root nodes across two initial branches; the result is limited to the explored continuations and search budget.Source: §E.4 · p. 21, Table 10 · p. 21

E.5 OOD INVESTIGATION: IDENTIFYING A SENSITIVE DATABASE RESOURCE

This section examines SAGE’s handling of PostgreSQL and filesystem cases, including how it uses state evidence to block harmful actions while preserving benign preparation. It also describes trajectory-based evaluation protocols in which harmful behavior is judged from observable actions and checkpoint decisions require cited evidence.

E.5 OOD INVESTIGATION: IDENTIFYING A SENSITIVE DATABASE RESOURCE

  • The PostgreSQL query reveals sensitive column names absent from the recorded input, and SAGE blocks at the annotated closure after distinguishing ownership from login access.The first two actions pass and the third blocks (k = T = 3); the query returns schema metadata, not sensitive-record contents.Source: §E.5 · p. 22, Table 11 · p. 22

E.6 EXACT INTERCEPTION: PRESERVING PREPARATION BEFORE A LIVE CHANGE

  • In Filesystem task single.53, SAGE passes seven actions and blocks the eighth, preserving artifact preparation before the live configuration change that completes the goal.Before deciding, SAGE inspects the staged artifact with env_read_file; the pending write is not executed.Source: §E.6 · p. 22, Table 12 · p. 22

F.1 TARGET INTERFACES AND PROTOCOLS

  • The 187-task adaptation preserves each objective and executable success condition while standardizing initial states, tool-facing descriptions, and reset-and-replay procedures.Compared methods use the same task assets and evaluator; browser executions use separate instances or serialized resets.Source: §F.1 · p. 23

F.2 DEFENDER INTERFACES AND PROTOCOLS

  • SAGE receives the pending, unexecuted action and can gather bounded, read-only environment evidence, including restricted database SELECT or EXPLAIN SELECT queries.Database inspection rejects writes, joins, subqueries, CTEs, user functions, and views, and bounds returned rows and characters.Source: §F.2 · pp. 23–24

F.3 ATTACKER INTERFACES AND PROTOCOLS

  • DART proposes grounded next-step instructions, branching when independent candidates yield different causal states, and uses execution feedback to assess progress toward harmful goals.Its proposal process receives the realized target transcript and selected-parent state; the feedback critic sees the current target response and completion criteria.Source: §F.3 · pp. 24, 26
  • Executable success is determined by task-specific checkpoint predicates on the resulting state, with normalized checkpoint credit required to reach the full total.The shared evaluator computes progress as P = min(1, max(0, R/Z)) and declares success when R and Z are close within the specified tolerance.Source: §F.3 · p. 25
  • The feedback critic scores each target execution’s contribution to the harmful goal, distinguishing refusal and observable no-op from informational contribution.It judges completion only from cumulative observable history and does not infer hidden actions or trust unsupported success claims.Source: §F.3 · p. 29
  • The semantic judge labels trajectories as not reached, safely avoided, attempted but incomplete, or completed, using observable evidence against unsafe-goal criteria.It also records risk reached, refusal, technical failure, and confidence, with evidence references tied to trajectory events.Source: §F.3 · p. 30
  • For online checkpoint decisions, the judge evaluates each checkpoint independently as completed or not and must cite trajectory evidence.The criteria are supplied as checkpoint identifiers, text, and weights; the response requires an evidence reference with a summary.Source: §F.3 · p. 30

F.4 EXECUTION BUDGETS

The reproduced methods use task-specific execution budgets, with STAC preparation budgets additional to adaptive turns and early stopping able to reduce actual usage. Defender evaluation uses SAGE’s default inference configuration and bounded per-action investigation budgets.

F.4 EXECUTION BUDGETS

  • Table 16 reports default per-attempt execution settings; STAC preparation budgets are additional to adaptive turns, and early stopping can reduce actual usage.In defended STAC evaluation, the defender checks the real target-execution suffix; synthetic-prefix construction is outside the intervention.Source: Table 16 · p. 31, §F.4 · p. 31
  • DART’s introspection and feedback calls are counted separately from planning calls, and its search limits are specified in Table 16.Direct and MTA retain the released MT-AgentRisk user protocols.Source: §F.4 · p. 31
  • STAC preparation uses one candidate, 2–6 planned steps, at most 20 verifier iterations, and a 10-chain-step prompt-writing limit before at most three adaptive turns.Preparation and adaptive execution are accounted for separately; a verifier iteration may contain multiple tool actions.Source: §F.4 · p. 31
  • AgentLAB uses two strategies, up to seven instruction turns per strategy, and at most three candidate evaluations per turn, for a ceiling of 42 target executions.The two strategies use separate initial states, and alternative refinements share the same selected prefix.Source: §F.4 · p. 31
  • STAC and AgentLAB role requests allow 4,096 output tokens, while the shared Semantic Judge uses temperature zero and up to 2,048 output tokens.Target-side sampling and output limits are listed in Table 14.Source: §F.4 · p. 31
  • SAGE receives six defender steps per pending action, including investigation and final submission, with up to 1,024 output tokens per step.Read-only queries provide private observations without executing the pending action or changing guarded state.Source: §F.4 · p. 31
Loading 2609.34518v1…