PaperlayerView exact revision on arXiv

Source-linked AI summary

Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

Huihan Liu, Changyeon Kim, Bo Liu, Minghuan Liu, Yuke Zhu

arXiv:2603.03818v2cs.LGcs.AIcs.RO

TL;DR

Continual learning asks whether robot policies can learn new skills without catastrophically forgetting earlier ones, a question previously studied mainly in smaller behavior-cloning models. The paper empirically evaluates pretrained VLAs, replay, pretraining, and retained task knowledge, finding strong resistance to forgetting, preserved forward learning, and rapid recovery of seemingly forgotten skills. Its scope assumes previous-task demonstrations are not fully available during current-task learning.

  • Problem

    Continual learning in robotics remains underexplored for large pretrained VLAs, especially whether replay requirements established for smaller models still apply.

  • Method

    The paper conducts comprehensive experiments evaluating VLAs in sequential imitation learning, including replay-buffer effects, pretraining, knowledge transfer, and recovery of prior-task performance.

  • Results

    Pretrained VLAs are remarkably resistant to forgetting: Experience Replay often achieves zero forgetting with 2% of training data while maintaining strong forward learning.

  • Takeaways & Limitations

    Large-scale pretraining and small replay data may support continual learning for VLAs, while retained representations can be reused to recover prior skills.

  • Takeaways & Limitations

    The continual-learning setting assumes demonstrations from previous tasks are not fully available while learning the current task.

Abstract

from arXiv · show

Continual learning is a long-standing challenge in robot policy learning, where a policy must acquire new skills over time without catastrophically forgetting previously learned ones. While prior work has extensively studied continual learning in relatively small behavior cloning (BC) policy models trained from scratch, its behavior in modern large-scale pretrained Vision-Language-Action (VLA) models remains underexplored. In this work, we found that pretrained VLAs are remarkably resistant to forgetting compared with smaller policy models trained from scratch. Simple Experience Replay (ER) works surprisingly well on VLAs, sometimes achieving zero forgetting even with a small replay data size. Our analysis reveals that pretraining plays a critical role in downstream continual learning performance: large pretrained models mitigate forgetting with a small replay buffer size while maintaining strong forward learning capabilities. Furthermore, we found that VLAs can retain relevant knowledge from prior tasks despite performance degradation during learning new tasks. This knowledge retention enables rapid recovery of seemingly forgotten skills through finetuning. Together, these insights imply that large-scale pretraining fundamentally changes the dynamics of continual learning, enabling models to continually acquire new skills over time with simple replay. Code and more information can be found at https://continual-vlas.github.io/forget-me-not/

1. Introduction

Continual learning requires robot policies to acquire new skills while preserving previous behaviors, but prior robotics studies largely examined smaller or lightly pretrained models. This study finds that pretrained VLAs are unusually resistant to forgetting, with pretraining improving transfer and retaining recoverable prior-task knowledge.

  • Continual learning requires balancing plasticity for new tasks with stability for previously acquired knowledge.
  • Prior robotics research primarily studied small policies trained from scratch or with limited pretraining, where forgetting often required large replay buffers or specialized regularization.
  • Pretrained VLAs are remarkably resistant to forgetting compared with smaller policies trained from scratch.
  • Simple Experience Replay often achieves zero forgetting on VLAs with only 2% of training data, and can sometimes improve previously learned tasks.
  • Pretraining reduces forgetting while preserving strong performance on new tasks, avoiding the usual trade-off between knowledge preservation and forward transfer.
  • VLAs can retain prior-task knowledge despite apparent performance degradation, enabling rapid recovery after a few finetuning steps.

2. Preliminaries

The paper formulates continual robot learning as sequential behavior-cloning across tasks with incomplete access to earlier demonstrations, and describes replay as a way to retain prior-task samples. VLAs map visual and language inputs to action chunks using pretrained multimodal representations.

  • A robotic task is modeled as a finite-horizon Markov Decision Process with state and action spaces, transitions, horizon, and an initial-state distribution.
  • Each continual-learning task has its own initial-state distribution and goal predicate while sharing the environment dynamics and spaces.
  • The study uses expert demonstrations containing observation-action trajectories for each task and seeks one policy after sequentially observing tasks.
  • Previous-task demonstrations are not fully available when learning the current task, making retention of earlier behavior necessary.
  • Experience Replay stores samples from prior tasks and combines them with current-task data to approximate the empirical distribution of all tasks seen so far.
  • VLAs use visual observations and language instructions to predict action chunks through a pretrained vision-language backbone and action head.

3. VLAs are Surprisingly Resistant to Forgetting

The study evaluates pretrained and non-pretrained robot policies in continual learning across LIBERO task suites, finding that pretrained VLAs resist forgetting while retaining strong forward transfer. Experience Replay is particularly effective, and the trend is consistent across VLA architectures.

  • Evaluation: The evaluation reports average task success rate and negative backward transfer, with lower NBT indicating less knowledge loss after learning new tasks.NBT is computed from task success rates measured after successive stages of training.
  • Resistance to forgetting: Pretrained VLAs with Experience Replay achieve near-zero or even positive backward transfer across multiple LIBERO benchmarks while maintaining strong forward transfer.Positive backward transfer means later-task learning can improve performance on previously learned tasks.
  • Resistance to forgetting: Similar resistance to forgetting appears across Pi0 and GR00T N1.5 despite differences in architecture, parameter count, and pretraining recipes.The authors interpret this consistency as evidence that ER effectiveness is not specific to one architecture or pretraining mixture.
  • Replay efficiency: At a 2% replay buffer, pretrained VLAs retain NBT around 0.1–0.2, whereas non-pretrained baselines reach 0.4–0.5.The non-pretrained models require more than 20% replay data to achieve comparable NBT; at 0.2%, all policies show near-complete forgetting and absolute NBT becomes metric-limited.

4. Pretraining Plays an Integral Role in Improving Continual Learning Performance

Large-scale pretraining improves continual learning by reducing forgetting under small replay buffers while preserving forward learning. This benefit reflects stronger knowledge transfer than in models trained from scratch.

  • Pretraining and replay: Pretraining reduces forgetting, especially when replay buffers are small, with the gap from non-pretrained models increasing as buffer size decreases.At zero Negative Backward Transfer, forgetting is absent; values below zero indicate positive transfer to past tasks.
  • Pretraining and replay: With 1000 samples, all variants show little to no forgetting, but pretrained variants achieve higher success rates on new tasks while maintaining prior performance.The buffer represents about 20% of the full dataset.
  • Pretraining and replay: With 10 transitions, Pi0 from VL + Action forgets less than the other variants and partially recovers performance on tasks T2, T3, and T4.The buffer represents about 0.2% of the full dataset.
  • Forward and backward transfer: Pretraining maintains high forward transfer while mitigating forgetting, avoiding low forgetting caused by insufficient plasticity.Pi0 from VL + Action and Pi0 from VL consistently outperform the model trained from scratch at sample size 1000.
  • Forward and backward transfer: Knowledge transfer increases steadily for Pi0 from VL + Action and Pi0 from VL, whereas Pi0 from scratch grows more slowly.Knowledge transfer aggregates success across tasks and therefore distinguishes genuine retention from failure to learn new tasks.

5. VLAs Retain Knowledge that is Seemingly Forgotten

VLA performance degradation after learning new tasks does not necessarily mean task knowledge was erased. Component analyses and rapid recovery show that relevant knowledge can remain represented and be reused through finetuning.

  • Knowledge loss and recovery: Component swapping shows that knowledge loss is compartmentalized across the VL backbone and action head rather than monolithic.Swapping either component produces performance between the original and fully updated models.
  • Knowledge loss and recovery: Swapping the VL backbone causes a larger performance drop than swapping only the action head across all four task categories.This indicates that action-relevant information is more consistent across tasks, while backbone updates alter representations.
  • Knowledge loss and recovery: Task diversity correlates with degradation: LIBERO-10 shows the largest drop, while LIBERO-Object shows minimal degradation when swapping the action component.LIBERO-10 has more diverse visual backgrounds, whereas LIBERO-Object shares similar pick-and-place actions.
  • Knowledge loss and recovery: Pi0 reaches peak performance within a small fraction of finetuning steps, much faster than during initial training, indicating preserved and reusable task knowledge.BC-Transformer requires a similar number of steps as its initial training, suggesting greater erasure of task knowledge.
  • Knowledge loss and recovery: Across all benchmarks, Pi0 recovers peak performance in fewer than 10% of the original training steps, while BC-Transformer often needs comparable or more steps.Recovery efficiency is the ratio of recovery steps Tf to original training steps To.

6. Related Work

Prior continual-learning research has focused mainly on small models and has established severe forgetting and stability–plasticity trade-offs. Large pretrained VLAs extend foundation-model capabilities to robotics, but their retention and recovery dynamics remain insufficiently understood.

  • Prior continual learning: Most prior continual-learning studies examine relatively small models trained from scratch, where catastrophic forgetting and stability–plasticity trade-offs are prominent.Common mitigation strategies include regularization, distillation, replay, and architectural isolation.
  • Prior continual learning: Studies of large pretrained models show that pretrained representations can reduce, but not eliminate, forgetting and alter transfer and interference patterns.This evidence comes primarily from language-model continual learning.
  • Vision-language-action models: VLAs combine perception, language understanding, and control from large heterogeneous datasets, enabling transfer across tasks and embodiments.Open X-Embodiment, Octo, and OpenVLA motivate their use in lifelong robot learning.
  • Open questions: The mechanisms by which large pretrained VLAs retain or recover skills under continual finetuning remain poorly understood, including whether classical replay is sufficient.This work positions pretrained VLAs as a distinct continual-learning regime governed by pretraining and representation reuse.

7. Conclusion and Discussion

The paper concludes that pretrained VLAs are unusually resistant to forgetting and can support continual learning with simple replay. Its experiments emphasize pretraining, retained representations, and recovery as central to this behavior.

  • Conclusion: Pretrained VLAs are surprisingly resistant to forgetting, and comprehensive experiments examine pretraining and retention of task-relevant knowledge.The conclusion presents these findings as guidance for future continual-learning paradigms for VLAs.
  • Supporting analyses: The paper’s supporting material includes additional continual-learning results, full confusion matrices, replay-buffer analyses, experimental details, model details, and training hyperparameters.Appendix sections also cover other factors and per-task knowledge-transfer curves.

A.1. Confusion Matrix Results for Comparison

The confusion-matrix results report task success after sequential ER training and compare forgetting across policies and LIBERO benchmarks. Normalized NBT changes the interpretation of LIBERO-10 forgetting, showing stronger knowledge retention by VLA models at larger replay sizes.

  • Confusion-matrix setup: Each confusion-matrix entry reports success on task j after training through checkpoint i under Experience Replay across four LIBERO benchmarks.The complete results cover Pi0, GR00T N1.5, BC-Transformer, BC-Diffusion, and BC-ViT.
  • Metric interpretation: Normalized NBT sets complete forgetting to 1.0 regardless of initial success rate, excluding tasks with zero initial success.This variant scales each forgetting term by the task’s initial performance.
  • Metric interpretation: Absolute NBT can make GR00T appear to forget most at small buffers because its initial success rates are higher.The metric penalizes larger initial performance drops more heavily.
  • LIBERO-10 results: At buffer size = 10, all policies experience near-complete relative forgetting under normalized NBT.The comparison concerns LIBERO-10 results across the evaluated policies.
  • LIBERO-10 results: At buffer size = 1000, Pi0 and GR00T retain substantially more knowledge, with normalized NBT near zero or negative.Negative normalized NBT indicates performance improvement on earlier tasks.

B.1. ER Training Details

The experiments fix replay and task-ordering conditions across methods while documenting the task sequences and baseline training procedures. ER uses a balanced mixture of current-task and replayed data under a fixed per-task buffer.

  • Replay configuration: All methods use a replay buffer of 1000 transitions per task, approximately 15–20% of each full task dataset.Data are equally sampled from the replay buffer and current task, giving a 1:1 ratio.
  • Experimental controls: The same randomized task order is used for every method within each LIBERO benchmark.This controls task ordering in the comparisons.
  • Task orders: LIBERO-Spatial contains ten object-placement and interaction tasks, including bowl, food, drawer, stove, and cabinet operations.The listed sequence specifies the tasks used in the benchmark order.
  • Baselines: EWC trains on current-task data while penalizing changes to parameters considered important for earlier tasks.Parameter importance is estimated using the Fisher information matrix after each task.
  • Baselines: Sequential training carries model weights across tasks and continues finetuning on current-task data without replay.It is the simplest continual-learning baseline described here.

C. Study on Other Factors that Contribute to VLA’s Continual Learning Behavior

The study attributes VLA continual-learning behavior to both model size and pretraining-related factors, while finding little effect from the training objective. Scratch-trained larger models forget less than smaller variants, and changing the objective preserves similar performance.

  • Model size: Larger scratch-trained VLA models show near-zero forgetting, while smaller Pi0 variants have higher NBT.The result indicates that model size contributes to forgetting dynamics.
  • Training objective: Changing Pi0’s action-expert objective from flow matching to ℓ2 regression produces little difference in continual-learning performance.The architecture is kept unchanged in this ablation.

D. Per-Task Knowledge Transfer Curves: Pi0 vs. BC-Transformer

Per-task knowledge-transfer curves compare Pi0 and BC-Transformer across all four LIBERO benchmarks using finetuning and Learn First Time references. Success rates are normalized per task to assess recovery toward each task’s LFT peak.

  • Benchmark comparison: The per-task breakdown compares Pi0 and BC-Transformer across LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-10.Figures 11–14 provide the benchmark-specific curves.
  • Curve layout: Each task’s curves include Finetuning and Learn First Time, with Pi0 in the top subplot and BC-Transformer in the bottom.This layout supports per-task comparison of recovery behavior.
  • Normalization: A normalized value of 1.0 indicates full recovery of the task’s Learn First Time performance.All success rates are divided by that task’s LFT peak.
Loading 2603.03818v2…