PaperlayerView exact revision on arXiv

Source-linked AI summary

A GHOST in Long-Horizon Agents: Governance Hazard from Overlooked Safety Constraints across Turns

XinPeng Shen, Lan Zhang, Yixiao Huang, Haoran Cheng, Jiewei Lai, Leilei Chen, Haoxiang Deng

arXiv:2610.02664v1cs.AI

TL;DR

Long-horizon agents can resume tasks without users restating instructions, yet earlier safety constraints may no longer govern execution, risking severe harm. The paper formalizes and evaluates GHOST, introduces SCARBench and STAR-Guard, and reports GHOST on GPT-5.5 alongside no observed events with STAR-Guard under its evaluation.

  • Problem

    Long interaction histories can separate still-required safety constraints from resumed tasks, and violations may cause severe, irreversible harm and undermine user trust.

    Source: Introduction · p. 1

  • Method

    The paper investigates GHOST theoretically and empirically with SCARBench, then proposes STAR-Guard, combining historical semantic constraint restoration with deterministic pre-execution audit.

    Source: Introduction · p. 2

  • Results

    11.5% of SCARBench instances met the strict GHOST definition on GPT-5.5; STAR-Guard eliminated observed GHOST on GPT-5.5 under the evaluation setting.

    Source: Introduction · p. 2, Conclusion · p. 9

  • Takeaways & Limitations

    Restoration reduces unsafe proposals, while deterministic auditing prevents residual violations from reaching the environment.

    Source: Introduction · p. 2

    Context growth and compaction may affect access to historical constraints, but are not induced by an attacker.

    Source: Problem Definition: Governance Hazard… · p. 4

Abstract

from arXiv · show

Long-horizon agents are now playing an increasingly significant role in assisting humans with complex problem-solving. However, it is exactly their extended interaction history that introduces an underexplored execution-safety concern. Under benign interaction conditions, an agent may execute an action that violates a safety constraint specified many turns earlier. We term this failure mode Governance Hazard from Overlooked Safety Constraints across Turns (GHOST), which may cause irreversible damage. Our experiments reveal that GHOST events are not isolated cases: this failure mode, occurring precisely under benign interaction conditions, yields an occurrence rate of 11.5% on GPT-5.5. Furthermore, we theoretically show that if the residual conditional violation hazard along each safe prefix is bounded below by a non-summable sequence, the execution enters the hazard region almost surely. Leveraging this theoretical insight, we further propose STAR-Guard, a two-layer defense coupling historical semantic safety constraint restoration with pre-execution audit. STAR-Guard restores applicable safety constraints to reduce unsafe proposals, while its deterministic audit layer prevents residual violations from reaching the environment. Consistent with this two-layer design, we observe no GHOST events in our experiments under the GPT-5.5 setup.

Introduction

Long-horizon agents can resume tasks without users restating instructions, yet earlier safety constraints may no longer govern execution. The paper investigates this risk empirically and theoretically, then proposes a two-layer defense.

Introduction

  • Prior instruction-following methods lack reliable mechanisms for recovering older constraints, while agent-safety benchmarks mainly target attack-driven failures [Li et al., 2023, Ferraz et al., 2024, Hua et al., 2024, Zhang et al., 2025, Tavakoli et al., 2025, Robinette et al., 2026; Ruan et al., 2024, Debenedetti et al., 2024, Andriushchenko et al., 2025].The paper reports little to no safety improvement from these methods on its scenarios unless they are oracle-based.Source: Introduction · p. 1
  • Violating historical constraints may cause severe, irreversible harm and undermine trust; related failures include unauthorized email deletion, destructive database operations, and unintended file removal [Edwards, 2025, Sharwood, 2025, Willison, 2026, Bort, 2026].Source: Introduction · pp. 1–2
  • 12.4% Unsafe Completion and 11.5% strict GHOST rates occurred on SCARBench for GPT-5.5 under benign, non-adversarial conditions.The benchmark evaluation excludes malicious attacks.Source: Introduction · p. 2
  • Non-summable lower bounds on conditional entry hazards imply almost-sure eventual entry into a hazard region.The paper models safety-constraint violations as trajectories entering hazard regions.Source: Introduction · p. 2
  • STAR-Guard restores task-relevant historical constraints and audits proposed actions before environment execution; it reduced Unsafe Completion and strict GHOST rates to zero on GPT-5.5.The evaluation used five independent repetitions per model–condition pair.Source: Introduction · p. 2
  • SCARBench provides 412 matched condition instances from 103 base scenarios across six tool-use domains for evaluating historical safety-constraint recovery.Source: Introduction · p. 2

Related Work

The analysis formalizes how historical safety constraints can lose governance over resumed tasks and shows when residual hazard accumulates into unsafe execution or GHOST. It also motivates STAR-Guard’s combination of semantic restoration and pre-execution audit, against prior work that does not reliably recover the applicable historical constraint.

Related Work

  • Prior long-context and goal-drift studies examine weakened instruction use over time, while retrieval and self-refinement do not reliably identify which historical safety constraint applies [Deshpande et al., 2025, Li et al., 2025, Tavakoli et al., 2025, Robinette et al., 2026].Those studies usually evaluate task following and constraint following together; this paper isolates completed tasks whose still-required historical constraint no longer governs execution.Source: Related Work · p. 2, Problem Definition: Governance Hazard… · p. 4
  • Existing agent-safety benchmarks cover risky tool use, prompt injection, harmful execution, and unsafe planning [Ruan et al., 2024, Debenedetti et al., 2024, Andriushchenko et al., 2025, Yin et al., 2024].Policy-enforcement approaches generally assume the applicable policy is already available at decision time; SCARBench instead uses benign contexts of varying lengths to evaluate safe completion.Source: Related Work · pp. 2–3
  • The benign setting assumes a safe initial environment, a benign user, no adversarial instructions, unchanged user authority, and a still-valid constraint unless updated or revoked.GHOST is the matched outcome in which the agent safely completes with the constraint explicit but completes unsafely when it is available only in earlier history.Source: Problem Definition: Governance Hazard… · pp. 3–4

Theoretical Analysis: Hazard-Region Reachability under Residual Hazard

  • Context growth can weakly raise a lower bound on residual hazard under attention-dilution conditions, but is a possible modifier rather than an unconditional cause of failure.The context-conditioned corollary requires execution to remain above L0 and to continue presenting safety-critical opportunities.Source: Theoretical Analysis: Hazard-Region… · p. 4
  • A non-summable lower bound on conditional hazard along safe prefixes makes eventual hazard-region entry almost sure, without requiring independent outputs, tasks, or transitions.For paired executions, strict GHOST has a non-vanishing asymptotic probability when explicit safe completion and implicit task persistence remain bounded away from zero; it is almost sure in the special case α = γ = 1.Source: Theoretical Analysis: Hazard-Region… · pp. 4–5
  • The paired GHOST result requires explicit safe completion and implicit task persistence to remain bounded away from zero, alongside a persistent context-induced hazard lower bound.A sufficient mechanism is a context-induced recovery burden that raises implicit-condition hazard above the matched explicit-condition hazard by at least δ.Source: Theoretical Analysis: Hazard-Region… · p. 5
  • STAR-Guard restores applicable constraints from an external library into the current task context, making compliant proposals more likely without guaranteeing safety.Constraints are structured by scope, trigger, and rule; restoration is semantic and probabilistic, so unsafe proposals may remain.Source: Mitigation Approach: STAR-Guard · pp. 5–6
  • Its audit binds active constraints to the proposed action and execution state, repairing and re-auditing when possible or blocking actions when safety cannot be established.Under sound operation, proposals reach the environment only after applicable constraints are satisfied directly or through valid repair; failed proposals are mapped to ⊥.Source: Mitigation Approach: STAR-Guard · p. 6

Evaluation Criteria: SCARBench

SCARBench uses executable, environment-grounded matched conditions to measure GHOST events while varying history length and whether the still-required constraint is restated. It evaluates both task completion and safety across tool traces and final environment states.

Benchmark and matched conditions

  • SCARBench’s executable scenarios measure Safe Completion, Unsafe Completion, and GHOST events under matched short/long and explicit/historical-only conditions.The benchmark uses 103 unique scenarios across six tool-use domains, instantiated as 412 condition instances per target model; benign long contexts contain 6,000–6,176 tokens across 56–160 turns.Source: Evaluation Criteria: SCARBench · pp. 6–7

Notation and protocol

  • Each case is tested under four matched conditions: Short Explicit, Short Implicit, Long Explicit, and Long Implicit, varying history length and constraint availability at resumption.The task, safety constraint, user authority, and environment remain unchanged across conditions.Source: Evaluation Criteria: SCARBench · pp. 6–7

Outcome evaluation

  • For each run, Safe Completion requires both task completion and a safe tool trace and final state; failure of either safety check makes a completed task Unsafe Completion.Separate evaluators check task completion, trace compliance, and final-state safety, detecting unsafe intermediate procedures even if later repaired.Source: Evaluation Criteria: SCARBench · p. 7

Matched conditions

  • The difference-in-differences metric estimates whether historical constraint recovery reduces Safe Completion more under long history than under short history.A negative value indicates a larger reduction under long than short history.Source: Evaluation Criteria: SCARBench · p. 7

Operationalizing GHOST

  • GHOST is counted when an agent safely completes with an explicit constraint but completes unsafely when the same still-required constraint must be recovered from long history.The rate is the number of observed GHOST events divided by matched case–repetition pairs.Source: Evaluation Criteria: SCARBench · p. 7

Experiments

Experiments find strict GHOST across evaluated models and show greater Safe Completion degradation when historical constraints must be recovered under long histories. STAR-Guard improves safety and task completion, with its strongest overall trade-off in the reported Qwen3.5-4B ablation.

Experimental setup

  • The experiments assess GHOST across models and task/context settings, the long-history recovery penalty, and whether STAR-Guard reduces unsafe actions and GHOST.Each model–condition pair is evaluated over five independent repetitions unless otherwise stated.Source: Experiments · p. 7

Strict GHOST under benign long context

  • 11.5% strict GHOST occurs on GPT-5.5; across seven targets, all-case GHOST ranges from 6.8% on Kimi-K2.6 to 27.8% on Qwen3.5-4B.GPT-5.5’s Safe Completion falls 12.4 percentage points and Unsafe Completion rises 11.4 points when the constraint must be retrieved from long history.Source: Experiments · pp. 7–8

Long-history safety degradation

  • All seven evaluated models show negative long-history difference-in-differences values; Qwen3.5-4B has the largest additional Safe Completion loss at −36.7 percentage points.The results span API-served and locally deployed models; GPT-5.5’s value is −13.4 percentage points.Source: Experiments · p. 8, Figure 3 · p. 8

STAR-Guard evaluation

  • STAR-Guard increases Safe Completion by 7.2–34.2 percentage points and reduces Unsafe Completion by 8.3–29.2 percentage points across models.On GPT-5.5, Safe Completion rises from 76.3% to 94.0%, with no observed Unsafe Completion or GHOST events.Source: Experiments · p. 8, Table 2 · p. 8

Component ablations

  • On Qwen3.5-4B, Full STAR-Guard achieves 81.2% Safe Completion and 1.9% Unsafe Completion, while Audit Only has the lowest GHOST rate at 1.7%.Restoration alone raises Safe Completion from 47.0% to 76.3% and reduces Unsafe Completion/GHOST from 31.1%/27.8% to 4.3%/3.7%.Source: Experiments · p. 8, Table 2 · p. 8

Baseline comparison

  • On Qwen3.5-4B, STAR-Guard improves Safe Completion by 34.2 percentage points and reduces Unsafe Completion and GHOST by 29.2 and 25.9 points versus No Defense.LIGHT Three-Memory reaches 58.4% Safe Completion while retaining 23.7% Unsafe Completion; DVR reduces Unsafe Completion and GHOST to 0.4% but assumes a ground-truth verifier.Source: Experiments · pp. 8–9, Table 3 · p. 9

Conclusion

The paper identifies GHOST, formalizes its risk through conditional hazard-region reachability, and introduces SCARBench and STAR-Guard. Experiments report GHOST across seven models and no observed GHOST with STAR-Guard on GPT-5.5 under the evaluation setting.

Conclusion

  • The paper identifies GHOST, introduces a 412-instance benchmark across six tool-use domains, and proposes STAR-Guard, which substantially improves Safe Completion and eliminates observed GPT-5.5 GHOST under the evaluation setting.The paper formalizes risk through conditional hazard-region reachability.Source: Conclusion · p. 9

A Supporting Hazard Analysis

The analysis models safety violations as entry into hazard regions and bounds survival along safe prefixes using conditional entry hazards. A non-summable sequence of lower bounds implies almost-sure hazard-region entry, subject to the stated opportunity and context conditions.

Setup and notation

  • A violation occurs when a safety-critical opportunity enters the hazard region of prohibited actions or unacceptable resulting states.A prohibited trace prefix remains unsafe even if later repaired, while individually permitted actions can compose into an unacceptable state.Source: A Supporting Hazard Analysis · p. 11

Almost-sure entry

  • If conditional entry hazards on safe prefixes have non-summable lower bounds, hazard-region entry occurs almost surely.The proposition frames this as a conditional hazard–survival safety obligation.Source: A Supporting Hazard Analysis · p. 11

Finite-horizon bound

  • A uniform positive residual hazard lower bound yields a finite-horizon bound that approaches one exponentially in the number of safety-critical opportunities.The bound concerns safety-critical opportunities, not raw dialogue turns.Source: A Supporting Hazard Analysis · p. 11

Context-conditioned corollary

  • The context-conditioned result assumes that after a threshold context length, residual hazard stays bounded below on every safe prefix.It does not assume that increasing context length universally increases hazard.Source: A Supporting Hazard Analysis · p. 12
  • If context remains above the threshold and safety-critical opportunities continue indefinitely, hazard-region entry occurs almost surely conditional on the stated event.The conclusion need not follow if context resets, constraints are reinforced, or the lower bounds become summable.Source: A Supporting Hazard Analysis · p. 12

A Sufficient Attention-Dilution Mechanism

The section presents attention dilution as a conditional route from longer contexts to residual safety hazards, not an unconditional cause of failure. It also distinguishes almost-sure hazard entry from GHOST, which additionally requires task completion to persist after a violation.

A Sufficient Attention-Dilution Mechanism

  • Under stated conditions, increasing context can reduce attention to a visible safety span, but softmax normalization alone does not prove that attention declines.The conditions are stylized because added context may also change the query and key representations.Source: A Sufficient Attention-Dilution… · p. 12
  • Attention affects execution risk only under an additional behavioral assumption connecting safety-span attention to a lower envelope on environment-facing entry hazards.The paper does not treat attention itself as compliance or use attention as a causal explanation for matched behavioral results.Source: A Sufficient Attention-Dilution… · p. 12
  • If residual hazards have a non-summable lower bound along safe prefixes, eventual task-level violation occurs almost surely; positivity alone is insufficient.The cited product argument requires more than every opportunity having positive hazard.Source: A Sufficient Attention-Dilution… · p. 13
  • Strict GHOST additionally requires explicit-condition Safe Completion and persistent task completion under the implicit condition; when α = γ = 1, it occurs almost surely as opportunities grow.The paired result links hazard-region entry to GHOST rather than equating every violation with GHOST.Source: A Sufficient Attention-Dilution… · p. 13
  • Context length can affect hazard convergence only through an empirically justified change in residual hazard; context growth is a possible modifier, not an unconditional cause.Hazard accumulation alone does not establish GHOST because the event also requires task completion to persist.Source: A Sufficient Attention-Dilution… · p. 14

B SCARBench Construction and Protocol

SCARBench evaluates recovery of still-valid safety constraints across matched short/long and explicit/implicit task-resumption conditions. Its executable cases define task completion, trace safety, and final-state safety, with paired long-history conditions sharing the same history.

B SCARBench Construction and Protocol

  • The evaluators distinguish safe completion from unsafe completion using task completion plus both trace and final-state predicates.Unsafe intermediate actions count even if later actions repair the final state; unsafe final states also count.Source: B SCARBench Construction and Protocol · p. 14
  • Each canonical case has a distinct frozen benign history stream reused unchanged for its paired Long Explicit and Long Implicit instances; the suite has 103 such histories and 412 total condition instances.The reported 6,000–6,176 tokens and 56–160 turns describe each intervening history, excluding the initial and resumed requests.Source: B SCARBench Construction and Protocol · p. 14
  • Each case is evaluated in four matched conditions: Short Explicit, Short Implicit, Long Explicit, and Long Implicit, varying history length and whether the constraint is restated.The resumed task remains explicit and identifies itself as a continuation; corresponding condition pairs hold task wording or long-history content fixed.Source: B SCARBench Construction and Protocol · pp. 14–15
  • The representative-case table binds requests, tool arguments, environment objects, and evaluator targets to the same concrete path.This keeps the case’s target consistent across its executable components.Source: Table 5 · p. 15
  • SCARBench has 103 canonical cases across six tool-use domains, with 18 calendar/device cases and 17 cases in each of five other domains.Each case includes a target, prerequisite, sensitive action, safe reference trace, and forgetful reference trace.Source: B SCARBench Construction and Protocol · p. 15

Representative Executable Case

The representative filesystem case contrasts resuming a file-deletion task with and without restating its backup prerequisite, amid a frozen benign history. Its evaluator checks completion, action ordering, and the final backup state.

Representative Executable Case

  • The case holds the benign history fixed and changes only the final resumed request: Long Explicit restates the backup constraint, while Long Implicit requires recovering it from earlier context.The history contains 84 shared benign messages and 6,038 benign tokens.Source: Representative Executable Case · pp. 15–16
  • The frozen history’s unrelated exchanges include discussion of Kyoto literature, Japanese writing, and Bauhaus-influenced buildings.The listing presents excerpts from the beginning, middle, and end of the history, with omitted messages retained in the case files.Source: Representative Executable Case · pp. 15–16
  • The Long Explicit request asks to delete temp/experiment_data.csv after creating a backup, while the Long Implicit request omits that prerequisite.Both requests resume the earlier task.Source: Representative Executable Case · p. 16
  • Completion requires an effective delete call and absence of the target file in the final state.The target path is bound consistently across the case’s requests, tools, environment, and evaluator.Source: Representative Executable Case · p. 16
  • Trace safety requires a successful backup of the same path before any attempted deletion, and state safety requires that backup to remain afterward.A failed prerequisite does not establish safety, and a schema-valid dangerous call reaching the handler remains an unsafe attempt even if the tool errors.Source: Representative Executable Case · p. 16

Outcome Taxonomy and Statistics

The outcome taxonomy separates successful safe execution, completed but unsafe execution, safe non-completion, unsafe attempts, and execution failures. The section also cautions against interpreting conditional safety among completed tasks as a primary measure or treating a difference-in-differences statistic as evidence of internal forgetting.

Outcome Taxonomy and Statistics

  • Safe Completion requires a completed task with safe trace and final state; Unsafe Completion is completed work with an unsafe trace or state.Safe Non-completion, Unsafe Attempt, and Execution Failure distinguish incomplete runs by hazard crossing and meaningful safe progress or explicit safety decisions.Source: Outcome Taxonomy and Statistics · p. 16
  • The five outcome categories sum to 100%, and conditional safety among completed tasks is not a primary metric because incapability or indiscriminate blocking can inflate it.The full-denominator taxonomy keeps incomplete outcomes in view.Source: Outcome Taxonomy and Statistics · p. 16
  • The 95% GHOST intervals use 10,000 case-cluster bootstrap resamples, retaining repeated observations and both matched conditions for sampled cases.This preserves within-case dependence across repetitions.Source: Outcome Taxonomy and Statistics · p. 16
  • A negative Safe Completion difference-in-differences value indicates a larger long-history loss for implicit constraint recovery, but does not identify an internal forgetting mechanism.The interpretation concerns the matched long-versus-short and implicit-versus-explicit contrast.Source: Outcome Taxonomy and Statistics · p. 17

C Experimental Protocol

The protocol evaluates resumed tasks with historical constraints, while STAR-Guard stores applicable rules, restores them when scope and trigger match, and audits proposed actions before execution.

C Experimental Protocol

  • The target-agent loop uses role-separated history, shared tool schemas and a stateful environment, with up to five iterations and five runs per model–condition pair.Each iteration is limited to 512 output tokens; temperature is 0.7 where available, and No Defense receives no auxiliary safety anchor.Source: C Experimental Protocol · p. 17

D STAR-Guard Details

  • Incoming messages are checked for persistable constraints, which are compiled into validated lifecycle rules and stored for later activation.At a resumed task, a separate semantic gate activates rules only when both scope and trigger match, then restores them to the current request.Source: D STAR-Guard Details · p. 17
  • STAR-Guard audits proposals before execution, blocking matching prohibitions and repairing unmet prerequisites only when no prohibition applies.Every action introduced by a repair plan is audited through the same boundary; the guarantee is action-local, not unconditional end-to-end safety.Source: D STAR-Guard Details · pp. 17–19

End-to-End Example

  • For prerequisite rules, repair performs a verified same-target action before rechecking the proposal; a different target cannot authorize it.The filesystem example requires a successful backup of the same file before deletion; prohibition rules instead block the prohibited action without repair.Source: D STAR-Guard Details · p. 17, End-to-End Example · p. 19
  • The ablations separate restoration in the prompt from tool interception: Full STAR-Guard combines both, while Restoration Only and Audit Only isolate each component.No Defense uses the target agent without restoration or interposition.Source: End-to-End Example · p. 19

E Baseline Configurations and Failure Analysis

The section compares baseline interventions with STAR-Guard and explains why retrieving or retaining history alone may not ensure constraint-compliant execution.

E Baseline Configurations and Failure Analysis

  • Prompt Reminder, BM25 and DeCRIM have distinct failure modes: they may encourage action without the right prerequisite, miss paraphrases or long-history constraints, or mistake method-control text for requirements.A missed historical constraint cannot then become a reliable critique criterion.Source: E Baseline Configurations and Failure… · p. 20
  • 56.9% SC for VerIFY-Summarize is 9.9 points above No Defense, but UC rises from 31.1% to 35.3% and GHOST remains 27.4%.LIGHT Three-Memory reaches 58.4% SC, with 23.7% UC and 18.6% GHOST; the passage attributes residual failures to memory not guaranteeing target-bound execution order.Source: E Baseline Configurations and Failure… · p. 20
  • TrustAgent and DVR are oracle-information comparators: TrustAgent receives the correct constraint, while DVR receives the exact executable verifier, so neither tests online constraint capture.TrustAgent reports 5.2% UC and 4.7% GHOST; DVR reports 75.1% SC, 0.4% UC and 0.4% GHOST.Source: E Baseline Configurations and Failure… · p. 20
  • Under an information boundary where source-turn ingestion cannot observe later resumed requests or subsequent history, STAR-Guard achieves 81.2% SC and 1.9% for both UC and GHOST in the Qwen3.5-4B comparison.Source: E Baseline Configurations and Failure… · p. 21

F Prompt Inventory

  • The prompt inventory specifies the target-agent and baseline prompts, while STAR-Guard processes each complete incoming message without benchmark labels or later-turn information.The target agent receives the resumed request as the live prompt, with prior role-separated messages as history; baseline memory methods use summaries, retrieved turns, or decomposed constraints.Source: F Prompt Inventory · p. 21, Target-Agent Prompt · p. 21, STAR-Guard Prompts · p. 22

Related papers

Loading 2610.02664v1…