Source-linked AI summary
Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, Sergey Levine
TL;DR
VLAs must preserve semantic knowledge from pretrained VLMs while producing fast continuous actions, but naïvely adding action experts can harm training dynamics and knowledge transfer. The paper combines discrete-action representation learning, VLM co-training, and a gradient-insulated continuous action expert, showing that this recipe mitigates the degradation across robot-control evaluations.
Problem
It remains unclear whether continuous diffusion or flow-matching action experts preserve pretrained VLM knowledge and how they affect VLA training dynamics.
Method
The method jointly trains discrete action and language prediction with a continuous flow-matching action expert, co-trains on VLM data, and blocks expert gradients from reaching the pretrained backbone.
Results
The proposed method consistently achieves the highest performance in the reported real-world evaluations while improving language following, convergence speed, and knowledge transfer.
Takeaways & Limitations
Insulating the VLM backbone while adapting it through discrete actions provides an effective recipe for continuous-action VLAs.
Takeaways & Limitations
Training both continuous and discrete outputs increases computational cost by about 20%, and language following remains imperfect.
Abstract
from arXiv · showhide
Vision-language-action (VLA) models provide a powerful approach to training control policies for physical systems, such as robots, by combining end-to-end learning with transfer of semantic knowledge from web-scale vision-language model (VLM) training. However, the constraints of real-time control are often at odds with the design of VLMs: the most powerful VLMs have tens or hundreds of billions of parameters, presenting an obstacle to real-time inference, and operate on discrete tokens rather than the continuous-valued outputs that are required for controlling robots. To address this challenge, recent VLA models have used specialized modules for efficient continuous control, such as action experts or continuous output heads, which typically require adding new untrained parameters to the pretrained VLM backbone. While these modules improve real-time and control capabilities, it remains an open question whether they preserve or degrade the semantic knowledge contained in the pretrained VLM, and what effect they have on the VLA training dynamics. In this paper, we study this question in the context of VLAs that include a continuous diffusion or flow matching action expert, showing that naively including such experts significantly harms both training speed and knowledge transfer. We provide an extensive analysis of various design choices, their impact on performance and knowledge transfer, and propose a technique for insulating the VLM backbone during VLA training that mitigates this issue. Videos are available at https://pi.website/research/knowledge_insulation.
1 Introduction
VLAs adapt pretrained VLMs for robotic control but must reconcile discrete-token representation learning with continuous, real-time action generation. The paper proposes knowledge insulation to preserve backbone learning while using a separate continuous action expert.
- VLAs extend VLMs to control robots by combining end-to-end robotic learning with knowledge transferred from web-scale vision-language pretraining.
- The approach uses discrete action representations for fast representation learning and continuous actions from a smaller expert for fast, precise inference.
- Robotic observations and control require architectural changes because physical systems provide inputs such as multi-view images, proprioceptive states, and continuous actions.
- Continuous adapters improve control efficiency but can disrupt VLM training and degrade language interpretation and overall policy performance.
- Knowledge insulation trains the backbone with discretized actions and VLM data while training a flow-matching action expert without sending its gradients into the backbone.
2 Related work
Prior VLA research adapts pretrained VLMs for generalizable robot control and introduces specialized continuous-action decoders for fast inference. These additions can slow training and reduce transfer of web-pretrained knowledge.
- Multi-modal large language models: Multimodal architectures can separate modality-specific experts that cross-attend, a design explored here for integrating robot actions into pretrained VLMs.
- Vision-language-action models (VLAs): VLAs fine-tune pretrained VLMs for action prediction and can transfer web-scale knowledge to improve policy generalization.
- Fast and continuous action decoding mechanisms in VLAs: Diffusion heads and flow-matching action experts provide continuous outputs and fast inference by adding new weights and losses during VLA fine-tuning.
- Fast and continuous action decoding mechanisms in VLAs: These added weights and losses often make VLAs slower to train than autoregressive counterparts and reduce transfer from web data.
- π0 and π0.5 models: The π0 recipe uses a continuous action expert for efficient dexterous control, but experiments show degradation in language following and training speed.
3 Standard vision-language-action (VLA) model training recipes
Standard VLA training adapts multimodal transformers to predict robot actions from visual, language, and state inputs. Action chunks may be represented discretely for autoregressive prediction or continuously with diffusion and flow matching.
- A VLA predicts real-valued robot actions conditioned on multi-view images, proprioceptive state, and a natural-language instruction.
- Action representations: Action chunking represents a future trajectory of robot actions relative to the current robot state.
- Action representations: Naïve discretization converts every action dimension in a chunk into tokens, framing robot action prediction as next-token prediction with cross-entropy loss.
- Action representations: FAST compresses action information over time with a discrete cosine transform to reduce token counts and computational cost.
- Action representations: Diffusion and flow matching train on noised continuous action chunks and generate final actions by denoising or integrating the predicted flow.
- State representations: Robot state can be represented as discretized text, discretized special tokens, or a continuous state passed through a learned projection.
- VLA architectures, training, & mixture of experts: VLA architectures typically use pretrained multimodal transformers to map multimodal input tokens to action or language output tokens.
4 Problems with standard VLA recipes
Standard VLA recipes trade off inference speed, training speed, language following, and transfer from VLM pretraining. Autoregressive decoding is slow, continuous-action modules can disrupt the backbone, and freezing the backbone is insufficient for robotics.
- Autoregressive VLAs are slow: Autoregressive VLAs limit action resolution and require slow sequential inference; π0-FAST takes approximately 750 ms to predict a 1-second action chunk on an RTX4090.
- Robotic specific architectures and modality adapters don’t benefit as much from VLM pretraining: Robotics-specific modules enable faster control, with π0 reaching 10 Hz compared with 1.3 Hz for autoregressive VLAs.
- Robotic specific architectures and modality adapters don’t benefit as much from VLM pretraining: Those robotics-specific adaptations benefit less from VLM pretraining because the new modules are not initialized with pretrained weights.
- VLM pretraining does not have sufficient representations for robotics— freezing doesn’t work: Freezing pretrained VLM weights yields 0% performance because the frozen representations lack sufficient robotics-specific information for highly performant policies.
- In the spoon-bussing example, π0 ignores the command, π0-FAST eventually succeeds slowly, and the proposed recipe solves the task with fast inference and rapid convergence.
5 Improving VLAs with co-training, joint-training & knowledge insulation
The proposed recipe combines discrete and continuous action prediction, co-trains on heterogeneous data, and insulates pretrained backbone weights from action-expert gradients. These choices support faster training and reduce interference while retaining continuous actions for inference.
- Joint-training: The action expert provides continuous actions for fast inference, while autoregressive objectives are used during training to accelerate representation learning.The smaller action expert is retained for test-time continuous control, and the autoregressive objective is not required for inference.
- Joint-training: The recipe jointly trains autoregressive language and discrete action prediction with flow-matching continuous action prediction.The model can produce continuous action chunks while using discrete prediction as a training-time representation-learning objective.
- VLM data co-training: The model combines VLM, action-only, and language-annotated action data through masked losses for different modalities.The loss masks determine where language and action objectives apply, allowing heterogeneous examples to be mixed during co-training.
- Joint-training: The attention mask prevents discrete FAST action tokens from attending to continuous action tokens and vice versa.This design keeps the discrete and continuous action representations from directly attending to each other during joint training.
- Knowledge insulation: Knowledge insulation stops gradients from the flow-matching action expert from updating pretrained backbone weights.The restriction targets interference from newly initialized action-expert parameters while allowing the backbone to learn action representations through discrete outputs.
- Knowledge insulation: The modified attention computation separates backbone-backbone, action-to-backbone, and action-expert interactions while applying stop-gradient to selected pathways.The attention probabilities are decomposed into Pbb, Pab, and Paa components, and stop-gradient restricts gradient flow through the chosen computation.
6 Experiments
Experiments compare the proposed method with VLA baselines across real-world manipulation, generalization, convergence, language following, and representation-learning settings. The method consistently performs strongly while preserving language following and improving training efficiency relative to flow-matching-only training.
- Experimental setup: The evaluation covers dexterous, long-horizon manipulation across multiple robot embodiments, including table bussing, shirt folding, items in drawers, mobile manipulation, and LIBERO.Held-out scenes are used for items-in-drawer and mobile-manipulation evaluations.
- Task performance and baselines: The proposed method consistently achieves the highest performance in real-world evaluations, including the held-out items-in-drawer task.Joint training, π0, π0-FAST, and HybridVLA show task-specific weaknesses, including language-following failures, slow movement, or imprecise drawer opening.
- Benchmarks: 0.55 ± 0.09: the proposed method’s DROID score exceeds π0 at 0.49±0.09 and π0-FAST at 0.45±0.09.The method also achieves state-of-the-art performance on LIBERO-90 and LIBERO-Spatial but performs worse on LIBERO-10.
- Convergence speed: 7.5 times as many training steps: π0 requires this amount to reach performance similar to the proposed method on the generalist table-bussing evaluation.The proposed method trains as quickly as π0-FAST, while π0 converges more slowly under flow-matching loss alone.
- Generalization: Co-training on VLM data is particularly important for semantic generalization to novel objects in mobile manipulation.Removing VLM data leads to slightly worse task completion in the generalist evaluation, while the method trained with VLM data performs best in unseen environments.
- Modeling choices: FAST provides a better representation-learning signal than naive tokenization, while sub-sampling naive tokens with stride 5 outperforms dense naive tokenization.Naive tokenization remains better than training with continuous actions alone but worse than FAST.
7 Discussion & Limitations
The paper finds that naïve continuous-action VLA training can substantially damage pretrained VLM knowledge, and proposes knowledge insulation to mitigate this degradation. The method improves training efficiency and language following, but adds training computation and does not eliminate language neglect.
- Discussion: Knowledge insulation shields the pretrained VLM backbone from gradients produced by the continuous action expert while retaining discrete-action learning signals.The continuous flow-matching expert is trained alongside discretized actions, allowing robot-task adaptation without backpropagating its gradient into the VLM.
- Discussion: Experiments across real-world and simulated tasks support the claim that naïve training loses pretrained knowledge and that knowledge insulation mitigates this degradation.
- Limitations: Training with continuous and discrete outputs increases computational cost by about 20%, but faster convergence still yields shorter wall-clock training than purely diffusion-based VLAs such as π0.
- Limitations: Language following improves but remains imperfect because correlations in training data may still cause the model to ignore instructions.
A.1 Common public benchmarks
The evaluation covers simulated LIBERO and real-world DROID benchmarks, using specialist models trained on each benchmark's data. LIBERO also includes a generalist model trained on diverse robot and non-robot data without prior LIBERO examples.
- Benchmark coverage: The study evaluates specialist models on both the simulated LIBERO benchmark and the real-world DROID benchmark.
- Generalist evaluation: A LIBERO generalist model is fine-tuned from a model trained on diverse robot-embodiment and non-robot data that excluded LIBERO during original training.
- DROID: DROID uses real-world tabletop tasks involving picking and placing, wiping, and opening or closing drawers, with unseen environments and objects.Each trial is scored by task progress, and results are reported as the average score.
- LIBERO: LIBERO evaluation uses four task suites—Spatial, Object, Goal, and Long—and jointly trains one policy across them.LIBERO-Long corresponds to LIBERO-10, while LIBERO-90 is evaluated separately with a policy trained only on that dataset.
A.2 Real-world tasks
Real-world evaluation uses tabletop manipulation tasks, with performance measured over repeated episodes and statistical significance assessed using a two-sided t-test.
- Evaluation protocol: Each real-world task is evaluated with 10 episodes per task per policy.
- Evaluation protocol: Performance is reported according to the task-specific scoring procedure, with statistical significance evaluated using a two-sided t-test.
A.2.1 Tasks with static robots
The static-robot evaluation spans drawer organization, T-shirt folding, and table bussing, using task-specific progress scores. Specialist models are trained on matching robot embodiments, while table bussing also includes a diverse-data generalist.
- Items in drawer: The items-in-drawer task awards points for opening the drawer, placing one item inside, and closing it, for a maximum of 5 points.The specialist uses data from static single-arm robots.
- T-shirt folding: The T-shirt-folding task scores the final fold's squareness and wrinkle amount, with a maximum score of 5.The specialist uses data from static bimanual robots.
- Table bussing: The table-bussing task requires placing 12 objects into the correct utensil/dish or trash receptacle, awarding one point per correct placement.The maximum score is 12, and both specialist and diverse-data generalist models are evaluated.
A.2.2 Tasks with mobile manipulator robots
The evaluation uses four household tasks for a mobile bimanual robot, each scored by task-specific subgoals. The generalist model was trained on diverse robot embodiments and broader vision-language tasks than those evaluated here.
- Evaluated tasks: Make bed requires straightening a blanket and placing two pillows at the head of the bed, for a maximum score of 5.The initial state has a partially unmade bed with two gray pillows at its foot.
- Evaluated tasks: Dish in sink requires placing four dishes or utensils into the sink, for a maximum score of 8.One point is awarded for picking up an item and one for placing it in the sink.
- Evaluated tasks: Mobile items in drawer requires picking up an item, opening and closing the drawer, and placing the item inside, for 4 points.
- Evaluated tasks: Laundry in basket requires navigating to and picking up clothing, placing it in or on the basket, and getting it fully inside, for 3 points.
- Training data: The generalist model is trained across 12 robot configurations and a diverse task set extending beyond the four evaluation tasks.Examples of additional robot tasks include grinding coffee beans and hanging a towel on an oven handle.
- Training data: Its general VLM training data includes image captioning, visual question answering, and object localization with added indoor-scene and household-object annotations.
B Training details
Training combines a pretrained PaliGemma VLM backbone with a smaller action expert that processes noisy action chunks and predicts vector fields. The setup specifies asymmetric information flow, low-timestep sampling, and three proprioceptive-state representations.
- Model architecture: The model uses pretrained PaliGemma weights as the VLM backbone and a smaller transformer action expert processing noisy action sequences.
- Model architecture: For action horizon H = 50, the expert projects noisy actions into its embedding dimension and injects timestep information through an MLP and adaptive RMSNorm.
- Model architecture: The action expert outputs action tokens that a final linear projection decodes into the target vector field.
- Model architecture: The 2B VLM backbone uses width 2048 and depth 18, while the action expert uses width 1024 and has 300M parameters.
- Information flow: Information flows from the VLM prefix to the action expert, while the expert cannot attend to FAST action tokens to prevent representation leakage.
- Training choices: The timestep distribution emphasizes low timesteps rather than uniform or midrange sampling.
- State representations: The study compares text, special-token, and continuous representations for the robot proprioceptive state q ∈ R^s.
- State representations: Text state resembles pretrained natural text but uses the most tokens, whereas special-token and continuous states introduce randomly initialized inputs.