PaperlayerView exact revision on arXiv

Source-linked AI summary

When Is a Memory Safe Enough to Act On? Action-Conditioned Verification for Long-Lived Agents

Yiqi Wang, Yifei Wang, Jiaqi Zhang, Zhangkai Wu, Yiqun Duan, Taotao Cai, Mingkai Zheng

arXiv:2610.31446v1cs.MA

TL;DR

Long-lived agents must decide whether remembered premises are reliable enough for a specific action, since the verification need depends on the action and the cost of error. The paper formulates verification as a stopping problem and builds AMV-GUARD to select checks by bounded value of information. On ToolSandbox, it lowers verification cost relative to a static policy while retaining reported safety and safe-proceed outcomes; its AppWorld transfer carries trade-offs and does not migrate the full pipeline.

  • Problem

    Memory improvement methods address stored entries, but whether to re-check a memory before acting depends on the candidate action and the cost of being wrong.

    Source: §1 · pp. 1–2

  • Method

    AMV-GUARD extracts action premises, calibrates beliefs about memory errors, and uses bounded value-of-information selection in a verification gate.

    Source: §1 · p. 2

  • Results

    33.9% lower mean verification cost relative to AMV-Static, with 100% safe-proceed recall and 0% Gate harmful on ToolSandbox.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    The reported findings support setting verification effort by the consequences of the action and whether checking can change the agent’s next decision.

    Source: §7 · p. 9

    Simulator harm rates cover only scored consequences, excluding reputational damage, downstream human cost, and delayed effects.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Long-lived agents act on memory carried across sessions, and many of their actions are irreversible, so a stale memory can turn directly into harm. Existing work improves the memory itself --- retrieving, refreshing, or reconciling entries --- yet whether an entry must be re-checked before acting is not a property of the entry: the same remembered address can be used unchecked for a routine shipment but should be re-confirmed before a valuable one. We therefore study \emph{action-conditioned memory verification}: given a candidate action, decide which of its memory premises to re-verify and when to proceed or abstain, balancing execution risk, abstention, and verification cost. We formulate this as a sequential stopping problem, in which a verification is worth its cost only if it can change what the agent does next; under suitable verifier reliability and cost conditions, increasing action consequence can move the same memory through proceed, verify, and direct-abstention regimes. Building on this formulation, \textsc{AMV-Guard} is a practical verification gate that extracts the premises a candidate action rests on, calibrates beliefs about whether they are wrong, and selects checks by bounded value of information (VOI). On ToolSandbox, relative to a static policy given the same inputs, it lowers mean verification cost by 33.9\% while keeping 100\% safe-proceed recall and 0\% Gate harmful, and raises Proceed coverage from 38.83\% to 44.17\%; in a gate-core transfer to AppWorld, which carries over the stopping/VOI core but not the full structured pipeline, it lowers verification cost by 21.3\% at a small cost in safe-proceed recall and task success. These results suggest that verification effort should be set by what an agent is about to do, not only by how reliable a memory appears.

1 Introduction

Memory-side signals alone cannot determine whether to verify: the same memory can warrant proceeding, checking, or declining depending on the action and the cost of error. The paper formulates this as action-conditioned verification and introduces AMV-GUARD and its empirical evaluation.

1 Introduction

  • The same stored address warrants different decisions for routine and valuable shipments because the actions have different costs of error.For the valuable shipment, the agent should re-confirm the address and decline if it cannot be confirmed.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Existing memory-focused, grounding, and information-acquisition approaches do not jointly account for the action and how stored premises support it.An action-independent score cannot distinguish the two shipment cases.Source: §1 · p. 2
  • Action-Conditioned Memory Verification asks which premises of a candidate action to verify, when to execute, and when to decline, accounting for asymmetric error loss and verification cost.The formulation treats verification as a stopping problem because a check is valuable only if it can change the decision.Source: §1 · p. 2
  • The paper formalizes action-specific verification demand and shows that, under suitable verifier reliability and cost conditions, increasing consequence can move a memory through proceed, verify, and decline-without-verifying regimes.It separates relevant, action-supporting, and critical memories and treats abstention as a terminal decision.Source: §1 · p. 2
  • AMV-GUARD combines premise extraction, calibrated memory-error beliefs, and bounded value-of-information selection in an executable verification gate.The empirical study measures verification-cost gains, adaptive-verification conditions, transfer trade-offs, and failure boundaries.Source: §1 · p. 2

2 Problem Formulation

The formulation separates candidate-specific safety decisions from the memory claims supporting them, and treats verification as a sequential choice among checking, proceeding, and abstaining. Its value comes from changing the terminal decision, not merely reducing uncertainty about a fixed action.

2.1 Setup

  • A candidate commit rests on premises required for executability, correct parameterization, and permission; verifiers may cover several premises or none.The candidate-local terminal choices are proceed and abstain, while invalidated premises return the candidate to the agent loop for possible replanning.Source: §2.1 · p. 3

2.2 Three notions the literature merges

  • Relevant memories are retrieved, action-supporting memories support a candidate premise, and critical memories are those whose error materially changes execution loss.These sets nest from critical to action-supporting to relevant, so verification budget should focus on the inner set.Source: §2.2 · p. 3
  • Under suitable cost and reliability conditions, increasing the consequence of a false premise moves the optimal initial decision from proceed to verify and then abstain.With an imperfect verifier the high-consequence region is finite; a perfect verifier supports verification at arbitrarily high consequence only when its cost is below the avoidable abstention loss.Source: §2.2 · p. 3

2.3 The decision problem

  • The candidate-local stopping value screens execution by expected-loss and harm-probability limits, and compares admissible execution with the abstention cost.The per-candidate screens govern whether proceeding is admissible; they are distinct from the empirical Gate harmful metric.Source: §2.3 · pp. 3–4
  • The exact information-constrained reference evaluates sequential verification using the deployable Gate’s information, rather than hidden truth.Its scope is exactness within the finite modelled problem, not a guarantee of premise completeness or correct calibration and observation models.Source: §2.3 · p. 4

2.4 Why expected loss reduction for a fixed action is insufficient

  • For a fixed action, Bayesian information does not reduce average action loss, so the naive loss-reduction VOI is nonpositive and a positive-value stopping rule never verifies.The verification value must be based on whether information can change the terminal decision.Source: §2.4 · p. 4
  • One-step VOI can miss complementary checks whose value appears only when taken together.The proposed bounded two-step lookahead considers one further affordable verifier, but is not the full Bellman policy.Source: §2.4 · p. 4

3 AMV-Guard

AMV-GUARD sits between the planner and executor, assessing the premises of each proposed commit and choosing whether to verify, proceed, abstain, or return an invalidated candidate for replanning. It combines premise extraction and calibrated risk with bounded, adaptive value-of-information selection.

3 AMV-Guard

  • AMV-GUARD identifies premises, forms calibrated beliefs over them, and verifies, proceeds, or abstains for each proposed commit.An invalidated premise returns the candidate to the agent loop; replanning is not a terminal VOI action.Source: §3 · p. 4, Figure 2 · p. 5

3.1 From retrieved memories to action premises

  • Premise extraction combines LLM proposals grounded to memory, verifier, or public action targets with tool-contract rules and a counterfactual filter.The implementation uses simplified coupling assumptions rather than general joint inference over arbitrary dependencies.Source: §3.1 · p. 5

3.2 From memory confidence to calibrated, consequence-aware risk

  • The system estimates claim-error probabilities from recency, source, consistency, graph position, and model signals, then converts them to action-precondition violation beliefs and consequence-aware stopping values.Model-reported confidence is a feature rather than a probability, and the loss encodes the asymmetric consequence groups.Source: §3.2 · p. 5

3.3 Adaptive greedy verification with a two-step fallback

  • At each step, the policy chooses the highest positive one-step VOI query, uses a positive two-step fallback when enabled, or returns the stopping decision if neither has positive value.After each observation it updates the posterior and recomputes scores, making this a bounded receding-horizon approximation rather than globally optimal planning.Source: §3.3 · pp. 5–6

4 Evaluation Protocol and Oracles

The evaluation uses replayable, perturbed instances and candidate-fixed interventions to assess action-premise safety. It distinguishes a structural target oracle from an information-constrained exact decision reference that uses deployable Gate inputs rather than hidden truth.

4.1 Replayable instances

  • Replayable instances are created by snapshotting a session state, applying a seeded hidden perturbation, and freezing the agent’s first commit as the candidate.Memory contains only facts observed through tools or supplied by the user, and variants of each base scenario share a split.Source: §4.1 · p. 6

4.2 Candidate-fixed safety interventions

  • Candidate-fixed safety labels come from flipping individual premises and, when needed, premise pairs to test whether execution outcomes change.The procedure does not re-run the planner or use planner-reexecution labels as Gate supervision.Source: §4.2 · p. 6

4.3 Two oracles, and what each measures

  • The structural target oracle audits dependencies and supports extraction scoring, while ExactPlanner-PredictedDep is a decision-agreement and regret reference using deployable information.The exact planner does not supervise the main method; its determinism establishes exactness only within the finite modelled problem, not premise completeness or model correctness.Source: §4.3 · p. 6

5 Experiments

The experiments evaluate AMV-GUARD’s safety–cost trade-off, the effects of action and consequence conditioning, adaptive verification, transfer, and failure boundaries. On ToolSandbox, it reduces verification cost versus static AMV while retaining safe-proceed recall and zero measured Gate harm.

5.1 Setup

  • The evaluation uses executable environments with state-based evaluators: ToolSandbox has 3,520 frozen instances, and AppWorld has 100 state-mutating tasks under clean and stale-memory conditions.ToolSandbox safe-proceed recall counts ground-truth-safe, Gate-evaluable original candidates executed during the initial Gate episode; verification cost accumulates across Gate entries in an episode.Source: §5.1 · pp. 6–7

5.2 RQ1: Does action-conditioned verification improve the safety–cost trade-off?

  • 33.9% lower mean verification cost than AMV-Static, with 100% safe-proceed recall, 0% Unsafe Pass, 0% Gate harmful, and 5.33 points more Proceed coverage.The main point also reports 73.91% task success, 7.03% raw end-to-end harm, and 1.219 mean verification cost. Across three backbones, adaptive VOI reduces mean verification cost by 33.9%, 42.3%, and 58.9%, while measured Gate harm remains zero.Source: Table 1 · p. 7, §5.2 · p. 7
  • Across three backbones, adaptive VOI lowers mean verification cost by 33.9%, 42.3%, and 58.9%, while coverage increases, stays unchanged, or decreases depending on the backbone.Coverage rises by 5.33 points for Qwen3.5-35B-A3B, is unchanged for Qwen3.5-27B, and falls by 5.68 points for DeepSeek-V4.1-Flash.Source: §5.2 · p. 7

5.3 RQ2: What do action conditioning, consequence, and premise structure each contribute?

  • In 100% of paired profiles, the high-risk action is verified and the low-risk action proceeds, with each selected query covering a premise of its candidate.The intervention holds the memory graph, impacts, verifiers, and Gate parameters fixed while changing the candidate action.Source: §5.3 · p. 8
  • With action and memory evidence fixed, AMV verifies 0%, 15.8%, and 62.8% of low-, medium-, and high-loss cases; high-loss harmful-proceed is 0% versus 64.2% without loss conditioning.Calibration lowers ECE/Brier/NLL from 0.319/0.319/4.401 for raw confidence to 0.082/0.209/0.588 with all fitting data, while retaining 100% safe-proceed recall and 0% Gate harmful.Source: §5.3 · p. 8

5.4 RQ3: When does adaptive multi-step verification help?

  • On a controlled subset, adaptivity lowers expected objective by 0.0314 and exact-planner regret from 0.0746 to 0.0431, at 0.257 additional mean verifier cost.The 6.8% multi-step rate bounds the observed gain to controlled conditions; enabling the two-step fallback raises safe-proceed recall from 17.0% to 100% and Proceed coverage from 5.31% to 31.25%.Source: §5.4 · p. 8

5.5 RQ4: Does it hold away from training conditions?

  • On AppWorld, AMV lowers verification cost by 21.28% versus AMV-Static but does not improve task success or safe-proceed recall.Task success is 28.00% versus 30.50%, and safe-proceed recall is 70.53% versus 72.63%; this transfers the decision core, not the full structured pipeline.Source: §5.5 · p. 8, Table 3 · p. 9

5.6 RQ5: Where does it fail?

  • On ToolSandbox, Gate harmful is 0% but raw end-to-end harm is 7.03% when planning and intent failures are included; AppWorld retains 7.89% Gate harmful.On ToolSandbox, 6.25% of episodes have an initial-candidate action or intent error, and 31.83% of Gate-evaluable instances resolve after feedback without a Gate-approved execution.Source: §5.6 · p. 8

6 Related Work

Related work improves memory stores, grounds agents in current environments, or applies cost-aware information acquisition. This paper distinguishes its setting by conditioning verification on the action and jointly handling multiple premises and verifier choices.

Long-term agent memory

  • Long-term memory work improves retrieval, consolidation, experience reuse, stale-memory handling, conflict resolution, and proactive surfacing, but computes action-independent quantities.The paper positions action-independent scoring as the gap its formalization and experiments address.Source: §6 · p. 9

Active grounding and environment probing

  • Environment-grounding methods query unstated context or structured world-model beliefs, whereas this work verifies persistent memory claims with provenance and allows declining.Its verifiers can differ in cost and reliability, cover multiple premises, and be selected in sequences.Source: §6 · p. 9

Cost-aware acquisition and selective prediction

  • The paper inherits classical cost-aware acquisition, stopping, and abstention machinery rather than claiming it, and applies it to multi-premise actions with overlapping, differently reliable verifiers.The stated distinction is adaptive selection of the next check in this setting.Source: §6 · p. 9

7 Conclusion

The conclusion reframes verification around whether a memory is reliable enough for the specific action and whether checking can change what happens next. ToolSandbox results show lower verification cost than a static policy without loss of safe-proceed recall, while broader validation remains open.

7 Conclusion

  • For an unchanged action, expected loss reduction alone gives no information benefit under Bayesian updating; verification is worth its cost when it can change the next decision.The conclusion reports that stopping-based VOI removes unnecessary checks.Source: §7 · p. 9
  • On ToolSandbox, AMV-GUARD lowers verification cost by 33.9% relative to a static policy without losing safe-proceed recall.Source: §7 · p. 9
  • Richer multi-premise dependency structure, longer verification sequences, and full cross-environment transfer of the structured pipeline remain to be validated.Source: §7 · p. 9

Limitations

The pipeline is bounded by model-driven premise extraction, approximations in its risk model, and consequence signals that may need to be specified by hand. The simulators also omit some real-world consequences and the paper reports no scalability or fine-grained breakdowns.

Limitations

  • Premise extraction cannot recover a premise never proposed, so extractor recall upper-bounds the pipeline.The counterfactual procedure filters proposed premises; RQ4 measures extraction under several OOD strata.Source: Limitations · p. 9
  • The risk engine cannot represent several distinct joint hyperedges in one candidate and approximates shared observations across covered premises.It uses one shared latent draw for jointly marked premises and charges loss when any represented loss group fails.Source: Limitations · p. 10
  • Loss asymmetries may require hand specification, while simulators omit reputational damage, downstream human cost, and delayed effects; scalability and detailed breakdowns are unreported.Reported harm rates therefore cover only modelled consequences.Source: Limitations · p. 10

Ethical Considerations

The paper frames AMV-GUARD as a safety gate whose deployment depends on how consequences are scored and whose coverage matters alongside safety. Its stopping rule distinguishes execution feasibility from the cost-sensitive value of further verification, and its theory permits verification demand to rise and then collapse as consequence increases.

Ethical Considerations

  • AMV-GUARD's harm rates cover only simulator-scored consequences, and reading low Gate-harmful rates as general safety guarantees could displace human oversight.Hand-specified loss asymmetries should be documented and audited; excessive declines also leave legitimate requests unserved, motivating coverage reporting alongside safety metrics.Source: Ethical Considerations · p. 10

A.1 Execution screens and the cost-sensitive objective

  • Under the cost-sensitive objective, execution thresholds determine whether PROCEED is admissible, but a cheap verification may still reduce expected terminal loss enough to justify its cost.This differs from the constrained objective, which stops as soon as execution is feasible.Source: §A.1 · p. 12

A.2 Consequences of Proposition 1

  • Below κ/p− verification has zero gross value, and below θ1 its value does not cover its cost, so memory-side uncertainty alone does not justify checking.These are stated as design constraints for the mechanism.Source: §A.2 · p. 12
  • With fixed memory, varying consequence can traverse proceed, verify, and decline-without-verifying regimes; verification demand can rise and then collapse.An imperfect verifier has an upper consequence level beyond which even confirmation cannot license execution; a perfect verifier remains worth using at arbitrarily high consequence only when c < (1 − p)κ.Source: §A.2 · p. 12

A.3 Proof

The proof compares stopping immediately with verifying once and then stopping, using posterior beliefs and asymmetric execution-versus-abstention losses. It derives where verification has no value, where its value exceeds cost, and how reliability and cost determine the upper consequence regime.

A.3 Proof

  • The proof sets terminal losses to θp for executing under a false premise and κ for abstaining, then defines the one-verification value from confirm/refute posteriors.The posterior probabilities p+ and p− follow from verifier reliability r and the observation likelihood π.Source: §A.3 · p. 12
  • Verification is worthless when θ ≤ κ/p− or θ ≥ κ/p+, because every posterior leads to the same execution or abstention choice as stopping immediately.In either range the verification cost c is incurred without changing the terminal decision.Source: §A.3 · p. 12
  • In the middle range, verification is optimal exactly when c < π(κ − θp+), a positive value that shrinks as θ rises and vanishes at κ/p+.This yields a verification interval that narrows as c grows and can become empty.Source: §A.3 · p. 12
  • For an imperfect verifier, the upper consequence cutoff is finite; with a perfect verifier it disappears only when c < (1 − p)κ.Otherwise verification cannot beat direct abstention at high consequence and is not strictly preferred there.Source: §A.3 · p. 12

B Method Details

The method details describe how AMV-GUARD represents premise dependencies, converts memory-error beliefs into action-precondition risks, and handles observations and replanning. They also specify computational costs, benchmark perturbations and oracle scope, and the comparison baselines.

B Method Details

  • The deployable risk engine represents multi-premise verifier coverage with one shared binary observation and does not consume distinct conjunction-only hyperedge identifiers.Jointly tagged premises share a latent draw, and loss is charged when any represented loss group fails; exact conjunction-only loss over multiple hyperedges remains future work.Source: B Method Details · p. 13
  • Memory-claim error probabilities are converted into action-precondition violation probabilities, while loss groups count their maximum active impact once rather than summing member impacts.The ToolSandbox consequence weights are fit for TaskFailure, WrongRecipient, and PolicyViolation; low/medium/high scales total action consequence, not separate irreversibility or collateral-damage analyses.Source: B Method Details · p. 13
  • After an observation, reaching invalidation threshold ρ returns REPLANREQUIRED rather than treating replanning as a terminal choice in the same VOI problem.The new candidate is evaluated as a fresh Gate entry using updated temporary session memory; immutable hidden state is unchanged.Source: B Method Details · p. 13
  • The one-step planner requires O(mk) posterior evaluations, while its conditional two-step fallback requires at most O(m^2k^2); exact dynamic programming is reserved for evaluation.The fallback runs only when all one-step scores are non-positive, and tool cost remains the primary experimental cost metric.Source: B Method Details · p. 13

B.2 Memory-error probability features

  • The calibrator uses recency, source, consistency, graph, and model-signal features, excludes high-cardinality identifiers, and selects calibration by held-out-family likelihood and calibration metrics.It fits a logistic base model and clips selected probabilities to [0.01, 0.99].Source: §B.2 · p. 13

C.1 Perturbation families

  • ToolSandbox perturbation manifests check family/variant/seed cardinality and other reproducibility and verifier bookkeeping; the dataset totals 3,520 instances split 2,240/640/640.The perturbation roster is described as implemented families, with counts designated train/development/test.Source: §C.1 · p. 14, Table 4 · p. 14

C.2 What the oracles do not certify

  • Exact dynamic programming is exact only within the experiment's predicted-premise set, binary observation model, verifier set, and budget; it does not certify premise completeness or model correctness.Deterministic replay alone does not guarantee complete premise enumeration, legal interventions, or correctly specified observations.Source: §C.2 · p. 14

D Baseline Roster and Adaptation Rules

  • AMV-Static uses the same predicted dependencies, calibrated beliefs, costs, and reliabilities as AMV-GUARD but does not reselect verifiers based on posterior updates.ExactPlanner-PredictedDep receives the same non-privileged inputs and is an evaluation reference, while thresholds are frozen after development-data selection.Source: D Baseline Roster and Adaptation Rules · p. 14

E Extended Results

Extended results report calibration and premise-extraction metrics, supplementary AppWorld and cross-backbone outcomes, and the study’s reproducibility, implementation, and evaluation boundaries.

E Extended Results

  • At 100% calibration data, the Gate reached 100% safe-proceed recall, 0% Gate harmful, 33.33% Proceed coverage, and 1.167 mean verification cost.Across 10/25/50/100% fitting data, calibrated ECE/Brier/NLL were 0.114/0.230/0.727, 0.123/0.230/0.697, 0.108/0.218/0.609, and 0.082/0.209/0.588.Source: E Extended Results · p. 15
  • Union+CF extraction scores vary by schema, with dependency accuracy dropping to 61.92% on unstructured language.Precision/recall/F1 is 100.0/100.0/100.0 on seen schemas, 100.0/95.91/97.91 on unseen tool families, 98.66/92.45/95.45 on unseen domain schemas, and 100.0/94.97/97.42 on unstructured language; dependency accuracy is 100.0%, 100.0%, 99.66%, and 61.92%, respectively.Source: E Extended Results · p. 15
  • AppWorld reports 2.11% Resolved without action beyond 57.37% Proceed coverage; ToolSandbox cross-backbone means use three seeds of 640 episodes each.At the reported ToolSandbox operating point, AMV-Guard coincides with ExactPlanner-PredictedDep; AppWorld’s paired-bootstrap intervals are broad given its 100-task transfer set.Source: E Extended Results · pp. 15–16

F Reproducibility

  • Test data were excluded from feature, calibration, threshold, and baseline-prompt selection, and environment replay is deterministic from snapshots and seeds.Large row-level premise files, per-step traces, and oracle policy trees are slated for release with the artifact.Source: E Extended Results · p. 17, F Reproducibility · p. 18

G License and Terms

  • ToolSandbox and AppWorld retain their upstream license terms; the planned author-written source release is Apache 2.0 and aggregate tables and figures are CC BY 4.0.Source: G License and Terms · pp. 18–19

H Intended Use and Responsible Release

  • The artifact is intended for research in simulated environments, not production safety certification; reported harm measures cover only evaluator-encoded consequences.All evaluated users and states are simulated and English-only, limiting representation of language, accessibility, cultural conventions, and harms faced by marginalized populations.Source: H Intended Use and Responsible Release · pp. 19–20

I Models and Computational Budget

  • Hosted APIs did not expose accelerator details or GPU-hours; total use is estimated at approximately 72,000 requests and 117 million tokens.The estimated ranges are 40,000–100,000 requests and 70–170 million tokens; total parallel wall-clock time is estimated at roughly 20–45 hours.Source: I Models and Computational Budget · p. 20

J Software and Implementation Details

  • At candidate commits, AMV-Guard extracts premises, calibrates violation beliefs, then selects positive-VOI verifications within budget or proceeds or abstains.Adapters intercept commits while retaining native benchmark evaluators; deterministic snapshot replay is added without changing those evaluators.Source: J Software and Implementation Details · pp. 21–22

Related papers

Loading 2610.31446v1…