PaperlayerView exact revision on arXiv

Source-linked AI summary

PredVLA: A Sub-Million-Parameter Predictive-Coding Policy for Robot Manipulation

Hiroki Sawada, Shunichi Kasahara

arXiv:2608.26673v1cs.RO

TL;DR

Large VLA systems achieve strong manipulation performance but leave open how much scale is necessary and whether alternative architectures can compete at much smaller budgets. PredVLA addresses this question with a sub-million-parameter predictive-coding policy whose latent state is updated online from sensory prediction errors. On LIBERO, it achieves strong multi-suite success and outperforms parameter-matched Transformer and LSTM policies under a controlled protocol, while enabling an exact open-loop condition for measuring observation-driven correction.

  • Problem

    It remains unclear how much model scale is necessary for strong language-conditioned control and whether alternative control architectures can remain competitive at much smaller parameter budgets.

  • Method

    PredVLA combines a frozen multimodal front end with a hierarchical recurrent generative model that predicts sensory signals and updates latent state online through prediction-error inference.

  • Results

    86.9% mean success across three short-horizon LIBERO suites and 75.4% across all four suites, with 3.7× and 7.4× the three-suite means of parameter-matched Transformer and LSTM policies, respectively.

  • Takeaways & Limitations

    A sub-million-parameter recurrent generative policy can achieve strong language-conditioned manipulation performance while providing an explicit mechanism for prediction-error-driven online state correction.

  • Takeaways & Limitations

    The study is limited to simulation, a single robot embodiment, the nominal LIBERO distribution, and separate controllers per suite; long-horizon performance remains lower.

Abstract

from arXiv · show

Large pretrained vision-language-action models dominate modern robot-manipulation benchmarks, but it remains unclear how much model scale is necessary for strong language-conditioned control, or whether fundamentally different control architectures can remain competitive at much smaller parameter budgets. We present PredVLA, a language-conditioned predictive-coding policy with only 0.68 million trainable network parameters and no robot-data pretraining, whose hierarchical generative recurrent dynamics predict visual features and proprioception while observations influence latent state only through online inference from the resulting sensory prediction errors. On LIBERO, PredVLA achieves an 86.9% mean success rate across the three short-horizon suites and 75.4% when the long-horizon suite is included. Under a controlled comparison using the same frozen front end, demonstrations, action decoder, and evaluation protocol, PredVLA achieves 3.7x and 7.4x mean success rates of parameter-matched Transformer and LSTM policies, respectively. The predictive-coding formulation also makes the contribution of observation-driven correction directly measurable: because observations influence the recurrent state only through prediction-error-based latent inference, disabling this inference yields an exact open-loop control condition. Together, these results show that a sub-million-parameter recurrent generative policy can achieve strong performance on modern language-conditioned manipulation benchmarks while providing an explicit mechanism for prediction-error-driven online state correction.

1 Introduction

PredVLA asks whether changing the control architecture can deliver efficient language-conditioned manipulation at a much smaller parameter budget. It introduces a predictive-coding recurrent policy and reports strong LIBERO performance under controlled parameter-matched comparisons.

  • Motivation: PredVLA addresses the cost of billion-parameter VLA systems by changing the control architecture rather than further compressing the prevailing paradigm.Existing compact VLA models remain in the hundreds-of-millions parameter regime.
  • Approach: Predictive coding uses generative dynamics to predict sensory trajectories and online latent-state inference to reduce discrepancies between predicted and observed signals.This formulation mediates action generation and sensory feedback through learned generative dynamics and prediction errors.
  • Approach: PredVLA pairs a frozen multimodal front end with a trainable hierarchical recurrent generative model that predicts visual features, proprioception, and a multimodal action distribution.Observed signals influence recurrent dynamics through online optimization of latent variables from sensory prediction errors and a learned-prior complexity term.
  • Results: 86.9% mean success across three short-horizon LIBERO suites and 75.4% including the long-horizon suite demonstrate strong performance from PredVLA.PredVLA uses 0.68M trainable parameters and no robot-data pretraining.
  • Controlled comparison: Under the same frozen front end, demonstrations, action decoder, and evaluation protocol, PredVLA achieves 3.7× and 7.4× the three-suite mean success rates of parameter-matched Transformer and LSTM policies.The comparison uses controlled behavior-cloning baselines.
  • Mechanistic analysis: The predictive-coding formulation supports mechanistic ablations and an exact open-loop condition by disabling online inference without changing learned language conditioning.The contribution passage also identifies hierarchical timescales and prediction-error-driven inference as performance-relevant factors.

2 Related Work

Related work spans compact VLA policies, deployment-time adaptation, predictive world models, and predictive-coding sensorimotor control. PredVLA differs by using fixed generative-model parameters with online latent inference and by evaluating this formulation on standardized language-conditioned manipulation benchmarks.

  • Compact VLA policies: Compact VLA models reduce computational cost but remain in the hundreds-of-millions parameter regime.TinyVLA, SmolVLA, and Evo-1 pursue compression through smaller or simplified architectures derived from prevailing VLA designs.
  • Online adaptation: Recent deployment-time methods adapt latent prompts, fast weights, or other components online to incorporate interaction history or correct predictions.The cited approaches differ in which quantities are optimized during deployment.
  • World-model-based policies: Large robot policies increasingly use future video, predictive visual representations, or shared video-action generation for action support and planning.These systems generally continue supplying current observations directly to the policy during execution.
  • PredVLA distinction: PredVLA places prediction inside online inference, using discrepancies between predicted and observed sensory trajectories to revise latent state that drives action generation.Prediction therefore updates the controller’s internal state rather than serving only as a representation, auxiliary target, or planning variable.
  • Predictive-coding foundation: Predictive-coding theory describes hierarchical top-down predictions and latent-cause revision through discrepancy minimization, with free-energy formulations balancing accuracy and complexity.This provides the theoretical and computational foundation for the predictive-coding formulation.
  • Prior predictive-coding control: Earlier predictive-coding recurrent models represented sensorimotor dynamics and inferred latent intentions from visual and proprioceptive prediction errors.PV-RNN enabled external observations to influence recurrent dynamics through backpropagated prediction errors rather than direct forward inputs.
  • Research gap: Prior predictive-coding work had not generally been evaluated as language-conditioned manipulation on LIBERO or against contemporary architectures at matched trainable parameter budgets.PredVLA brings predictive-coding recurrent inference into this modern VLA evaluation regime with fewer than one million trainable parameters.

3 Method

PredVLA combines a frozen multimodal front end with a trainable hierarchical predictive-coding recurrent controller whose observations affect latent state only through online prediction-error inference. The controller predicts visual features, proprioception, and actions while using hierarchical dynamics, lateral branch coupling, and free-energy minimization during training and deployment.

  • Frozen front end: PredVLA keeps language, vision, and proprioception preprocessing fixed while training only the recurrent controller.Language is encoded once per episode; two frozen ResNet18 camera encoders provide visual features, and proprioception is used directly.
  • Predictive-coding inference: Sensory observations influence recurrent state only through latent inference from visual and proprioceptive prediction errors, not through direct recurrent inputs.The efference-copy pathway carries predicted previous actions and proprioception, while observations enter the free-energy objective during inference.
  • Hierarchical dynamics: The recurrent controller comprises a root module T, visual branch V, and action hierarchy Atop→Abottom with multiple-timescale dynamics.T conditions both branches, V feeds Atop through a bottleneck, and module-specific time constants define the temporal hierarchy.
  • Free energy: Free energy combines dimension-averaged visual and proprioceptive prediction errors with a quadratic prior-complexity penalty over the evaluated sequence.The sequence is the training sequence during learning and a recent sliding window during online error regression; unavailable observations and padding are masked.
  • Training and inference: During training, posterior free variables and network weights are optimized jointly, whereas deployment freezes weights and updates posterior variables over a sliding observation window.The training objective additionally includes demonstrated-action likelihood, but that action term is excluded from test-time free-energy inference.
  • Open-loop control: Setting nitr = 0 produces an exact open-loop condition in which emitted actions are independent of visual and proprioceptive observations without retraining.Posterior variables remain equal to recurrently generated priors, while learned network weights and language conditioning remain unchanged.

4 Experimental setup

PredVLA is evaluated across four LIBERO suites under standardized preprocessing, training, inference, rollout, and controlled-baseline protocols. The experiments also test sensory corruption and architectural or inference-time ablations.

  • Benchmark and data: PredVLA is evaluated on SPATIAL, GOAL, OBJECT, and LONG, with ten tasks and 500 official demonstrations per suite.LONG demonstrations average 276 steps, compared with 125–149 steps for the short-horizon suites.
  • Preprocessing: The same shared PCA preprocessing is fitted jointly on demonstrations from all four suites and applied to PredVLA and every baseline.The PCA bases are fixed before policy training.
  • Model and optimization: The three short-horizon suites use one model configuration, while LONG widens recurrent time constants without changing the remaining architecture.All suites use the same optimizer and loss weights, but training sequence lengths differ between short- and long-horizon settings.
  • Evaluation protocol: Main evaluation uses 50 rollouts per task, 14 training seeds, 7,000 episodes per suite, and a 600-step rollout cap.Results are reported as means across the three short-horizon suites and across all four suites.
  • Ablations: Ablations use 20 rollouts per task and seven seeds; architectural conditions are retrained, whereas inference-time conditions reuse trained checkpoints.Observation-corruption tests replace a fraction α ∈ {0.25, 0.50, 0.75, 1.00} of one channel with observations from an unrelated episode.
  • Baseline comparison: Controlled baselines share PredVLA’s frozen front end, demonstrations, action head, and evaluation protocol, with Transformer and LSTM network sizes matched within 5%.Action-chunking Transformer variants use chunk lengths of 8 and 16 and have larger parameter counts.

5 Results

PredVLA delivers strong LIBERO performance at sub-million parameter scale and substantially outperforms controlled parameter-matched baselines. Ablations show that temporal hierarchy and online error regression matter, while sensory-channel effects vary across suites and tasks.

  • Main results: 86.94 mean success rate is achieved across the three short-horizon suites, versus 75.35 across all four suites.The four-suite scores are 83.19 on SPATIAL, 88.40 on GOAL, 89.24 on OBJECT, and 40.57 on LONG, using 675,732 trainable parameters.
  • Main results: 75.35 four-suite mean success rate exceeds 19.73 for the parameter-matched Transformer and 10.26 for the parameter-matched LSTM.The comparison uses the same frozen front end, demonstrations, action representation, and evaluation protocol.
  • Main results: 46 ms per step (21.6 Hz) is the reported end-to-end control latency on an RTX 5090, including the frozen front end and online error regression.Published OpenVLA and Octo results provide parameter-scale context but differ in pretraining, optimization, and evaluation protocol.
  • Architectural ablations: 18.66, 16.99, and 29.03 percentage-point drops follow replacement of hierarchical time constants on SPATIAL, GOAL, and OBJECT, respectively.This is the only architectural ablation consistently degrading performance across all three short-horizon suites.
  • Architectural ablations: 24.23 and 16.06 points are lost on SPATIAL and GOAL after removing the visual-to-action bottleneck, while OBJECT changes by only +0.04.Removing the efference-copy pathway causes smaller, suite-dependent reductions of 6.51, 11.84, and 3.31 points.
  • Online inference: 6.44, 9.56, 12.17, and 12.57 points are lost on SPATIAL, GOAL, OBJECT, and LONG when online error regression is disabled.Because observations enter recurrent dynamics only through error regression, this produces an exact open-loop condition.
  • Sensory error channels: 14.17 points are lost on OBJECT when proprioceptive error is removed, compared with 2.96 points for visual-error removal.On SPATIAL, the corresponding reductions are 3.51 and 2.80 points; on GOAL, they are 4.99 and 1.41 points.
  • Observation corruption: At α = 0.25, proprioceptive corruption reduces SPATIAL, GOAL, and OBJECT success by 5.71, 4.41, and 49.74 points, respectively.At α = 0.50, visual corruption changes success by less than five points on all three short-horizon suites.

6 Discussion

PredVLA demonstrates strong compact-policy performance under controlled comparison, while ablations clarify the roles of hierarchical dynamics and online sensory-error inference. The discussion also bounds these findings to the studied regime and shows that sensory channels matter differently across tasks.

  • 86.94% mean success across three short-horizon LIBERO suites and 75.35% across all four were achieved with 675,732 trainable parameters and no robot-data pretraining.
  • PredVLA outperformed parameter-matched Transformer and LSTM policies despite identical frozen front ends, demonstrations, action representations, and evaluation protocols.
  • The results do not establish predictive coding as generally superior because larger Transformer-based policies use different scales, pretraining regimes, and training settings.
  • 17–29 percentage-point losses followed replacement of hierarchical time constants with a common value across the short-horizon suites.
  • Disabling online error regression reduced success by 6–12 points on short-horizon suites and by 12.57 points on LONG.Because observations enter recurrent dynamics only through error regression, this ablation is an exact open-loop counterpart using the same learned controller and language conditioning.
  • Visual and proprioceptive prediction errors contribute differently across tasks, with OBJECT particularly sensitive to removing proprioceptive error.Similar suite-level averages can conceal different effects across individual tasks.

7 Conclusion

PredVLA is a compact predictive-coding policy that performs strongly on LIBERO under controlled comparisons and uses online prediction-error inference as an explicit correction mechanism. Its current scope remains limited to simulation, one embodiment, nominal tasks, and challenging long-horizon manipulation.

  • 86.94% mean success across three short-horizon LIBERO suites and 75.35% across all four were achieved with 0.68M trainable parameters and no robot-data pretraining.
  • PredVLA substantially outperformed parameter-matched Transformer and LSTM policies, as well as larger action-chunking Transformer variants, under the same evaluation conditions.
  • Ablations show that hierarchical recurrent dynamics and prediction-error-driven online inference both contribute to performance.Disabling inference provides an exact open-loop counterpart without changing the learned controller or language conditioning.
  • Visual and proprioceptive prediction errors contribute differently across manipulation tasks, indicating task-dependent use of sensory error signals.
  • The study is limited to simulation, a single robot embodiment, nominal LIBERO tasks, and separate controllers trained for each suite.Lower performance on LONG shows that long-horizon manipulation remains challenging for the current compact architecture.
Loading 2608.26673v1…