PaperlayerView exact revision on arXiv

Source-linked AI summary

When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models

Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev, Iuliia Tsvetkova, Egor Cherepanov, Nikita Kachaev, Artem Latyshev, Aleksandr I. Panov, Alexey K. Kovalev

arXiv:2610.22556v1cs.ROcs.LG

TL;DR

Long-horizon VLA policies may need visual cues absent from the current observation, while retaining full history is costly and redundant. RePlanMem-VLA uses revisions between overlapping action predictions to guide bounded-memory updates. The evaluated detectors achieve ROC-AUC of 0.981–0.997, and task-relevant observations persist to the later decision point in 100% of evaluated memory-retention episodes.

  • Problem

    Long-horizon manipulation can require historical visual cues that are absent from the current observation, but retaining full observation history increases context length and computation while adding redundancy.

    Source: §1 · p. 2

  • Method

    RePlanMem-VLA uses revisions between overlapping action predictions as an online signal for selecting keyframes and updating bounded visual memory, with filtering to suppress redundant frames.

    Source: §1 · p. 2, §3.5 · p. 6

  • Results

    ROC-AUC of 0.981–0.997 was obtained for selected-signal detectors, and task-relevant observations persisted to the later decision point in 100% of evaluated memory-retention episodes.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Plan disagreement can serve as a policy-internal signal for controlling which observations a VLA retains in bounded visual memory.

    Source: §6 · p. 10

    With execution stride H = 4, short phases may occur between replanning queries, while gradual transitions may spread revisions across queries without a pronounced peak.

    Source: §5 · p. 10

Abstract

from arXiv · show

Long-horizon, multi-stage Vision-Language-Action (VLA) policies often require visual information that is no longer present in the current observation. Retaining the complete observation history is computationally expensive and highly redundant, motivating selective visual memory. We introduce RePlanMem-VLA (Replanning-Guided Keyframe Memory for VLA), which turns a policy's own plan revisions into an online memory-write signal. Under receding-horizon execution, consecutive action chunks predict some of the same future actions. RePlanMem-VLA treats unusually large disagreement between these temporally aligned predictions as evidence of a task-relevant event. We first compare 35 disagreement measures across single-task autoregressive decoder-only and Flow-Matching Transformers trained from scratch, then validate the selected signals on pretrained OpenVLA-OFT and $\pi_{0.5}$ policies over MIKASA-Robo-VLA tasks. The resulting detectors achieve ROC-AUC of 0.981-0.997. We then use the selected signals in RePlanMem-VLA to trigger updates to a bounded visual memory, with a robot-masked write filter suppressing redundant frames. In the evaluated memory-retention experiments, RePlanMem-VLA preserves task-relevant observations until the later decision point in 100% of episodes. Real-robot replay data further show that event selection and redundancy suppression extend beyond simulation. These results establish plan revision as a policy-internal signal for controlling what a VLA remembers. Website: https://vika787.github.io/RePlanMem/.

1 Introduction

RePlanMem-VLA uses revisions between overlapping action predictions as an internal signal for selecting visual keyframes during replanning. Experiments assess disagreement measures and pretrained-policy transfer, while the memory mechanism filters redundant frames.

1 Introduction

  • Under receding-horizon execution, consecutive action chunks overlap in predicted future actions, and disagreement between aligned predictions signals that new visual information revised the policy’s intended behavior.The proposed hypothesis is that task-relevant observations induce unusually large revisions for the same future actions.Source: §1 · p. 2
  • RePlanMem-VLA derives its keyframe trigger from revisions between overlapping predictions already produced during normal replanning, then adds selected observations to visual context.A robot-masked write filter suppresses redundant frames; deployment requires neither keyframe annotations nor privileged simulator state.Source: §1 · pp. 2–3
  • ROC-AUC reaches 0.981–0.997 for selected-signal detectors on pretrained OpenVLA-OFT and π0.5 policies, and task-relevant observations persist to the later decision point in 100% of evaluated episodes.The study also reports real-robot replay evidence for event selection and redundancy suppression.Source: §1 · p. 2
  • The study screens 35 disagreement measures with decoder-only and Flow-Matching Transformers trained from scratch, then evaluates selected signals on pretrained OpenVLA-OFT and π0.5 policies.This tests the selected signals across the policy settings described in the introduction.Source: §1 · pp. 2–3
  • The bounded-memory mechanism uses plan disagreement to determine when to write and redundancy-aware filtering, including robot-masked clustering, to determine what to retain.The stated aim is to reduce redundant updates while preserving task-relevant observations.Source: §1 · p. 3

2 Related Work

Prior work studies memory representation and retrieval, keyframe selection, and plan consistency largely as separate problems. This paper connects them by using disagreement between overlapping action predictions to guide memory updates.

2 Related Work

  • Prior memory work addresses historical information through recurrent or belief-state latents, explicit memory, long-context conditioning, and retrieval- or structure-based approaches.These works study how historical information is represented or retrieved, while this paper focuses on when observations enter bounded memory.Source: §2 · p. 3
  • Existing keyframe selectors use motion or bottleneck-pose heuristics, learned frame scoring, progress or value signals, trajectory annotation, or dedicated event detectors.The passage characterizes these approaches as relying on a separate selector or external signal.Source: §2 · p. 3
  • Prior action-chunking work uses overlap to improve consistency or studies the trade-off between open-loop execution and replanning; this paper instead uses disagreement between overlapping predictions as a signal.The signal indicates that the new observation changed the policy’s intended behavior.Source: §2 · p. 3
  • The paper links memory representation, keyframe selection, and plan consistency by using plan disagreement as a policy-internal signal for memory updates.This contrasts with the related work’s separate treatment of historical-information representation or retrieval, external keyframe selectors, and overlap-based plan consistency.Source: §2 · p. 3

3 RePlanMem-VLA

RePlanMem-VLA uses revisions between overlapping action predictions to identify observations for a bounded visual memory. Its pipeline calibrates those revisions, filters redundant frames, and conditions final action predictions on any updated memory.

3.1 Overview

  • Plan revisions on overlapping action chunks provide an event signal for proposing keyframes before the final action prediction.The detector uses policy predictions rather than expert actions to drive memory writes.Source: §3.1 · p. 4, §3.3 · p. 4

3.2 Memory-augmented action policy

  • A bounded memory preserves earlier cues, while each policy query predicts K actions and executes only the first H before replanning.The memory is empty at episode start and is not carried across episode boundaries.Source: §3.2 · p. 4

3.3 Plan revision as an event signal

  • Comparing aligned predictions for the same future actions identifies revisions conditioned on the newly observed scene.The shared overlap has length K − H; a large discrepancy indicates that new visual evidence changed the policy’s intended behavior.Source: §3.3 · p. 4

3.4 Causal, task-conditioned event detection

  • Per-task causal normalization and surprise distributions calibrate a shared event threshold across tasks.The current sample is scored before it enters the running statistics, and calibration statistics are maintained separately for each task.Source: §3.4 · p. 5

3.5 Redundancy-aware frame selection

  • Complete-link clustering selects only candidate clusters with no frame already represented in memory, retaining each novel cluster’s most recent candidate.Visual masks suppress irrelevant appearance changes during comparison; the original RGB frames are stored and supplied to the policy.Source: Figure 5 · p. 5, §3.5 · p. 6

3.6 Event-gated memory update and second forward pass

  • Selected frames enter a fixed-capacity FIFO in chronological order, and an actual memory update triggers a second policy evaluation.If an event produces no selected frame, memory remains unchanged and no additional inference pass is required.Source: §3.6 · p. 7

3.7 Supervised fine-tuning objective

  • Expert actions supervise only the final control chunk; they do not identify events or select keyframes.The same causal detector and memory-update procedure can be used at inference without keyframe annotations or privileged simulator state.Source: §3.7 · p. 7

4 Experimental evaluation

The evaluation tests whether disagreement between overlapping action predictions can identify task-relevant keyframes and guide selective visual-memory updates. Across pretrained policies, detectors score highly; robot-masked redundancy filtering reduces writes while retaining key observations in the evaluated episodes.

4.1 RQ1: Disagreement signals in single-task policies

  • On phase-transition tasks, Flow-Matching Wasserstein-1 reaches Frame ROC-AUC 0.9388 and Energy distance reaches 0.9427; autoregressive Causal Z-score reaches 0.798/0.746 under full-context/current-only conditioning.Disagreement remains informative across context regimes and action-generation architectures, though effectiveness varies by task and setting.Source: §4.1 · pp. 7–8

4.2 RQ2: Event detection with pretrained VLA policies

  • ROC-AUC ranges from 0.981 to 0.997 and PR-AUC from 0.924 to 0.996 across pretrained-policy detectors; all three achieve event_recall@1 = 1.0.The detector inputs combine nine selected disagreement metrics with a visual-change feature.Source: Table 2 · p. 8, §4.2 · pp. 8–9

4 Experimental evaluation

  • The evaluation tests keyframe detection across policies and replanning regimes, then examines whether selected observations remain available until they are needed for action selection.The pretrained-policy study uses OpenVLA-OFT and π0.5 across 19 tasks under RH1 and EXEC4.Source: §4.2 · p. 8, §4.3 · p. 9

4.3.1 Direct surprise-gated writing

  • At the 95th-percentile threshold, cue-writing occurs in fewer than 20% of π0.5 episodes and none of the OpenVLA-OFT episodes; an 80th-percentile threshold yields 48% and 85%, respectively.Peaks often span multiple replanning steps, causing repeated writes for the same keyframe event.Source: §4.3.1 · p. 9

4.3.2 Redundancy-aware memory selection

  • Clustering raises OpenVLA-OFT cue-writing and retention to 100% in 50 episodes, but robot-pose variation can split the same environment state across clusters and cause redundant overwrites.In a bounded FIFO, such writes can evict task-relevant observations from earlier in the episode.Source: §4.3.2 · p. 9

4.3.3 Robot-masked redundancy filtering

  • With robot masking, OpenVLA-OFT memory writes fall from 336 to 169 and FIFO evictions from 186 to 19, while the keyframe is retained in 50/50 episodes.Masks are used for visual comparison; the original RGB observations are stored and supplied to the VLA.Source: §4.3.3 · p. 9

4.3.4 Transfer to real-robot replay

  • Real-robot replay shows detector responses at RememberColor phase transitions and robot masking suppressing redundant overwrites during the action phase.Figure 6 marks memory updates with white highlights and blocked overwrites with red crosses.Source: §4.3.4 · pp. 9–10, Figure 6 · p. 10

5 Limitations and Future Work

The paper identifies limits in interpreting plan disagreement, in the temporal structure of evaluated tasks, and in the memory and calibration designs.

5 Limitations and Future Work

  • Plan disagreement is not specific to semantic events: policy prediction errors can also produce revisions, so the signal indicates potentially important change rather than an unambiguous event.The experiments evaluate tasks with relatively clear phase boundaries; behavior under less distinct transitions remains uninvestigated.Source: §5 · p. 10
  • With execution stride H = 4, short phases may pass between replanning queries, while gradual transitions may spread revisions across queries without a pronounced peak.How disagreement behaves under less distinct phase transitions remains to be investigated.Source: §5 · p. 10
  • The current implementation stores selected observations as full images and uses visual clustering to suppress redundant entries.The authors identify more compact representations and comparison or aggregation mechanisms as future directions.Source: §5 · p. 10
  • Surprise statistics are task-conditioned and the threshold is selected empirically, so direct transfer to unseen tasks has not been shown without new reference statistics and threshold selection.The current formulation therefore requires calibration data for each task.Source: §5 · p. 10

6 Conclusion

The conclusion presents plan revisions as an internal signal for observation importance and for forming sparse visual memories in multi-stage manipulation.

6 Conclusion

  • Across single-task and pretrained policies, disagreement between overlapping action chunks signals task transitions and can guide selective memory updates, connecting replanning, event detection, and memory formation.The conclusion frames changes in intended future behavior as temporal proposals for sparse memory updates in partially observable, multi-stage manipulation.Source: §6 · p. 10

A Additional Analysis for RQ1

The appendix provides experimental setup, evaluation protocol, and supporting analyses for Section 4.1.

A Additional Analysis for RQ1

A.1 Experimental setup and reference events

The experiments screen disagreement metrics across eight single-task tasks and compare their keyframe discrimination on successful and failed trajectories. Task-specific results are reported by task, architecture, and context regime.

A.1 Experimental setup and reference events

  • Each single-task policy was evaluated on 50 closed-loop episodes, with action chunks of length 8 and one action executed before replanning.Reference event windows label both the final observation before a phase transition and the first observation after it as keyframes.Source: §A.1 · p. 14

A.2 Task-specific results

  • Table 4 reports the best-performing disagreement metric and its Frame ROC-AUC for each task, architecture, and context regime.The screening covers eight tasks, with task-specific reference keyframes defined from simulator phase-transition events.Source: §A.1 · p. 14, §A.2 · p. 15, Table 4 · p. 15

A.3 Disagreement signals on failed trajectories

  • On failed trajectories, median Causal Z-score ROC-AUC is 0.700/0.691 for autoregressive policies and Wasserstein-1 reaches 0.732/0.741 for Flow-Matching policies.The corresponding successful-trajectory values are 0.589/0.538 and 0.670/0.640; overall, failure does not systematically reduce disagreement-based discrimination.Source: §A.3 · p. 15

A.4 Correlation structure and metric redundancy

Disagreement measures contain substantial redundancy, especially among magnitude-based metrics, while temporal normalization captures a different aspect of plan revision. The correlation structure is similar across context regimes, motivating a compact feature set.

A.4 Correlation structure and metric redundancy

  • Magnitude-based measures, including point-wise, cumulative, and distributional metrics, show substantial redundancy; Causal Z-score and EMA deviation are less correlated with many of them.The authors suggest temporal normalization captures a different aspect of plan revision.Source: §A.4 · pp. 15–16
  • Full-context and current-only correlation matrices have Spearman correlations of 0.9647 for autoregressive policies and 0.9742 for Flow-Matching policies.Figure 7 aggregates within-episode Spearman correlations across eight tasks with equal task weights; rows distinguish architectures and columns distinguish context regimes.Source: §A.4 · pp. 15–16, Figure 7 · p. 16

B Additional Analysis for RQ2

  • The observed redundancy motivates using a compact feature set rather than retaining many closely correlated disagreement measures.The appendix also introduces extended results for the detector evaluation in Section 4.2.Source: §A.4 · p. 16, B Additional Analysis for RQ2 · p. 16

B.1 Experimental setup and evaluation

The detector evaluation uses episode-level data splits and validation-selected thresholds across pretrained policies and tasks. Within-episode examples display disagreement features alongside detector outputs for both policies and execution regimes.

B.1 Experimental setup and evaluation

  • Both OpenVLA-OFT and π0.5 were fine-tuned on the same 19 MIKASA-Robo-VLA tasks.The task set spans FindImposter, RememberColor, RememberShape, combined shape-and-color, and TakeItBack variants.Source: §B.1 · p. 16

B.2 Within-episode detection profiles

  • Figure 8 shows selected disagreement features and detector outputs in RememberColor5 episodes for both policies under RH1 and EXEC4.Panels include observations, within-episode standardized features, and keyframe probabilities from XGBoost, CatBoost, and LSTM detectors.Source: Figure 8 · p. 17, §B.2 · p. 17

B.1 Experimental setup and evaluation

  • For each task, policy, and execution regime, 50 closed-loop episodes were split by episode into approximately 70% training, 10% validation, and 20% test subsets.Detectors were trained separately for each policy and execution regime, with thresholds selected by validation F1.Source: §B.1 · p. 17

C Complete Registry of Disagreement Metrics

The registry describes disagreement signals computed from temporally aligned action sequences, including action differences, temporal features, plan-energy distributions, and action-component measures. It identifies nine action-derived features used in the later detector and memory system.

Complete Registry of Disagreement Metrics

  • Temporal features use first-action changes, with temporal differences and second differences defined over the sequence.The registry also includes cumulative drift, endpoint drift, and sign-flip measures.Source: C Complete Registry of Disagreement… · pp. 18–20
  • The registry marks nine action-derived features among the 35 disagreement signals as the features used in RQ2 and RePlanMem-VLA.The action sequences are aligned over an overlap of length L = K − H, with d components per action.Source: Table 6 · p. 18, C Complete Registry of Disagreement… · p. 18
  • Plan-energy quantities are defined by differences E = U − V and eE = eU − eV, indexed by i = 0, . . . , L − 1 and j = 1, . . . , d.Wasserstein-1 sorts independently within each action component; multiscale MMD uses B = {0.5, 1, 2}.Source: C Complete Registry of Disagreement… · pp. 18, 20
  • Action-component measures separately represent translation, rotation, and gripper disagreement.Translation cosine distance applies dead-zone handling: two below-threshold vectors have zero distance, while pairs with only one below threshold are excluded.Source: C Complete Registry of Disagreement… · p. 20
Loading 2610.22556v1…