Source-linked AI summary
Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models
Hongyu Shi, Sen Zhao, Zuyu Zhang, Lifeng Shen, Ding Zou, Xinyu He, Xu Zhang, Qinghua Zhang
TL;DR
FLOWMEM addresses the loss of successful latent reasoning after action generation by making it reusable as ordered reasoning flows. It recomposes and refines prior computation using current embodied context before action generation, and reports improved success on RoboMME and LIBERO-Plus.
Problem
Existing latent-reasoning VLAs discard successful computation after action prediction, while memory-augmented approaches do not retain how that reasoning evolved toward successful control.
Source: §1 · p. 1Method
FLOWMEM stores successful latent reasoning as ordered flows, recomposes compatible fragments, and refines the resulting route with current embodied context before action generation.
Source: §1 · p. 2, §3.4 · p. 5Results
48.0% versus 46.3% Full-16 average on RoboMME and 77.3% versus 73.2% pooled success on LIBERO-Plus for FLOWMEM and the no-memory policy, respectively.
Source: §4.2 · p. 7Takeaways & Limitations
The results support reusable latent reasoning flows as a practical mechanism for connecting successful prior computation to closed-loop VLA control.
Source: §6 · p. 9At evaluation, the policy can use only current embodied context and memory committed before the episode, not future observations, rewards, success labels, privileged states, or within-episode retrieval.
Source: §2.2 · p. 3
Abstract
from arXiv · showhide
Latent reasoning enables vision-language-action (VLA) models to transform multimodal observations into task-relevant internal states before generating continuous robot actions. While existing methods learn to generate or refine such states for each policy query, they discard successful reasoning after execution and therefore reconstruct similar computation from scratch. We present Reasoning and Flow Memory (FLOWMEM), a unified VLA model that turns successful latent computation into reusable reasoning experience. Rather than appending a fixed retrieved context, FLOWMEM dynamically retrieves and recomposes compatible latent fragments as the embodied context evolves, forming a reasoning route that follows the temporal structure and progress of successful computation. The route is then refined using current visual and proprioceptive evidence before it conditions action generation. Experiments on RoboMME and LIBERO-Plus show that FLOWMEM attains 48.0% and 77.3% success, outperforming memory-free policies by 1.7 and 4.1 percentage points, respectively. These results demonstrate the value of reusing successful latent computation for closed-loop VLA control.
1 INTRODUCTION
FLOWMEM treats successful latent computation as reusable, progress-ordered reasoning experience, addressing the loss of transient reasoning after action generation. It recomposes and refines prior reasoning for current control, with reported gains on RoboMME and LIBERO-Plus.
1 INTRODUCTION
- Latent reasoning is typically discarded after action generation, while existing memories preserve context without retaining how reasoning evolved toward successful control.This leaves successful reasoning computation unavailable for direct reuse at a familiar task stage.Source: §1 · pp. 1–2
- A latent reasoning flow is an ordered, progress-aware trajectory of computation, preserving how reasoning evolves rather than storing isolated vectors or historical policy context.A nearby latent point alone does not capture the direction or unfinished portion of computation.Source: §1 · p. 2
- FLOWMEM retrieves compatible transition fragments, recomposes complementary fragments in order, and corrects the resulting route using current embodied evidence before control.The route is compressed into fixed reasoning tokens and corrected by a fixed-depth residual block before action generation.Source: §1 · p. 2
- FLOWMEM combines an embodied context encoder, ordered reasoning-flow memory, a recomposition-and-refinement expert, and a memory-adaptive action expert in one latent-representation VLA.The action expert injects the refined route through the VLA’s native latent interface to produce executable action chunks.Source: §1 · p. 2
- RoboMME and LIBERO-Plus results show gains of 1.7 and 4.1 points, respectively, from treating successful latent computation as ordered reasoning experience.The evaluation covers memory-dependent manipulation and reuse under systematic distribution shifts.Source: §1 · pp. 2–3
2 PRELIMINARIES
Latent-reasoning VLAs insert an internal task-relevant state between multimodal context and action generation. The problem formulation uses successful episodes to construct memory-conditioned policies while keeping actions tied to current embodied context.
2.1 VISION-LANGUAGE-ACTION MODELS WITH LATENT REASONING
- A VLA’s latent-reasoning module produces an internal state from observation, robot state, and instruction, and its action expert uses that state to predict a future action chunk.The internal computation is normally discarded after action prediction.Source: §2.1 · p. 3
2.2 PROBLEM FORMULATION
- The reusable memory is constructed from successful training episodes, and the memory-conditioned policy uses compatible prior computation to improve expected task success.Predicted actions remain conditioned on the current observation and robot state.Source: §2.2 · p. 3
- Evaluation uses only current embodied context and pre-episode memory; future frames, rewards, success labels, privileged states, and within-episode retrieval are unavailable.This constrains reuse to remain closed-loop and responsive to the present physical state.Source: §2.2 · p. 3
3 THE PROPOSED MODEL
FLOWMEM stores successful latent reasoning as ordered fragments, recomposes compatible routes for the current embodied context, and refines them before action generation. Its memory pathway is integrated with the VLA’s native latent-conditioning and action-learning interfaces.
3.1 OVERVIEW
- FLOWMEM augments action prediction with memory M, retrieving ordered latent fragments that encode successful reasoning and refining route tokens using current embodied context.The refined tokens condition generation of continuous robot actions.Source: §3.1 · p. 3
3.2 EMBODIED CONTEXT ENCODER
- The Embodied Context Encoder represents the current observation, robot state, and instruction, while its retrieval query combines a visual descriptor, normalized state, and language.Only information available at the current policy query is used online.Source: §3.2 · p. 4
3.3 REASONING FLOW MEMORY
- Successful source episodes contribute successive-query latent reasoning states, partitioned into contiguous, ordered fragments with anchor, progress, transition, and provenance information.The stored payloads and metadata support later retrieval and recomposition.Source: §3.3 · p. 4
3.4 RECOMPOSE-AND-REFINE REASONING EXPERT
- Retrieval aligns fragments by visual semantics and state compatibility, retains ordered suffixes, and composes compatible transitions while preserving within-source order.Only latent reasoning fragments, not historical action segments, participate in recomposition.Source: §3.4 · p. 5
- Learned queries tokenize variable-length routes into fixed-size representations, which are refined using current embodied evidence; without a compatible route, the policy uses native latent reasoning.A fixed-depth residual operator reads current context as key-value evidence and corrects query-local route tokens.Source: §3.4 · p. 5
3.5 MEMORY-ADAPTIVE ACTION EXPERT
- The Memory-Adaptive Action Expert combines refined reasoning with current context through the VLA’s native latent interface to generate action chunks.Its conditional velocity field is integrated to obtain the action chunk.Source: §3.5 · p. 5
3.6 LEARNING OBJECTIVES
- Training aligns memory-conditioned reasoning tokens to frozen native targets while retaining the VLA’s original action-learning objective.The combined objective supervises reusable latent reasoning and executable actions.Source: §3.6 · p. 6
4 EXPERIMENTS
FLOWMEM is evaluated on RoboMME and zero-shot LIBERO-Plus, where it achieves the highest aggregate result among compared methods. Controlled analyses examine ordered multi-source reuse, memory properties, and current-context refinement.
4.1 EXPERIMENTAL SETUP
- RoboMME Full-16 uses 50 fixed episodes for each of 16 tasks, while LIBERO-Plus tests suite-trained policies zero-shot across seven perturbation domains.The LIBERO-Plus main result reports success separately for four original suites; its perturbation instances are reserved for zero-shot evaluation.Source: §4.1 · p. 6
4.2 MAIN RESULTS
- 48.0% on RoboMME Full-16 and 77.3% on pooled LIBERO-Plus are FLOWMEM’s highest aggregate results among compared methods.RoboMME improves from 46.3% with the controlled no-memory policy; LIBERO-Plus improves from 73.2%.Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7
4.3 EVIDENCE FOR LATENT-REASONING REUSE
- 45.6% for nearest-state retrieval and 44.0% for single-source retrieval contrast with 48.0% for FLOWMEM’s ordered multi-source recomposition on RoboMME.FLOWMEM composes compatible fragments across sources for 70.5% of policy queries; the comparison uses the same memory bank and fixed target episodes, policy interface, and action budget.Source: §4.3 · p. 7, Table 3 · p. 8
- 48.0% with correct memory exceeds wrong-task and wrong-progress memory by 1.9 points and out-of-order memory by 3.6 points.The largest drop follows disruption of temporal order.Source: §4.3 · p. 8, Table 4 · p. 8
- Removing multi-source recomposition lowers RoboMME success from 48.0% to 44.0%, while removing current-context refinement lowers it to 46.3%.On LIBERO-Plus, single-source and no-refinement variants reach 76.5% and 76.7%, respectively, versus 77.3% for the full model; variants retain the same action interface and latent-token budget.Source: §4.3 · p. 8, Table 5 · p. 8
5 RELATED WORK
Prior work has separately developed intermediate reasoning for VLA policies and memory for history-dependent control. FLOWMEM addresses their intersection by preserving successful latent reasoning as ordered flows that can be retrieved, recomposed, and refined.
RELATED WORK
- Reasoning-enhanced VLAs generate or refine intermediate computation for the current input, but do not retain successful latent reasoning trajectories as future experience.The cited approaches include explicit plans or visual goals, compressed latent planning, latent reasoning slots, and recurrent refinement.Source: §5 · pp. 8–9
- Robot-policy memories preserve textual task state, perceptual-cognitive evidence, observations, or recurrent policy state rather than the latent reasoning connecting observations to actions.These approaches address history-dependent control and include textual scratchpads, temporal retrieval, recurrent memory, and bounded visual experience.Source: §5 · p. 9
- Adjacent methods retrieve reasoning history, latent memory, trajectory priors, or subtask state, whereas FLOWMEM preserves ordered latent flows and recomposes compatible fragments from multiple episodes.FLOWMEM then refines the resulting route against the current observation before decoding it with the original action expert.Source: §5 · p. 9
6 CONCLUSION
FLOWMEM turns successful latent computation into reusable reasoning-flow memory. Its reported results improve main-task success over memory-free LaST0 on RoboMME and LIBERO-Plus.
CONCLUSION
- FLOWMEM stores ordered latent fragments with progress and transition metadata, retrieves anchor-aligned suffixes, recomposes compatible multi-source routes, and refines them using current embodied context.The resulting tokens are used before action generation.Source: §6 · p. 9
- FLOWMEM improves reported main-task success over memory-free LaST0 on RoboMME and LIBERO-Plus.Controlled retrieval and memory-counterfactual experiments are consistent with source compatibility, temporal order, and progress alignment contributing to performance.Source: §6 · p. 9
- The authors report AI assistance in planning, implementation, analysis, and manuscript preparation, with manual review against implementations, logs, and sources.They state that generative AI was not used to create synthetic datasets or establish theoretical or mathematical claims.Source: AI USE STATEMENT · p. 10
- The paper directs readers to Sections 3 and 4 for methods and protocols, table captions for metrics, and Appendix A for per-task and perturbation-wise results.Comparisons use stated protocols and fixed test episodes, with success rates aggregated from episode-level records.Source: REPRODUCIBILITY STATEMENT · p. 10
A.1 ROBOMME PER-TASK RESULTS
FLOWMEM’s 1.7-point Full-16 gain is spread across tasks rather than driven by a single outlier, though results vary by task.
A.1 ROBOMME PER-TASK RESULTS
- The 1.7-point Full-16 gain is distributed across 10 of 16 tasks, with 34 positive points and 6 negative points.The per-task results use the official 16-task, 50-episode-per-task protocol.Source: §A.1 · p. 12, Table 6 · p. 12
- The largest gains are on Visual Unmask (40.0% vs. 34.0%), Visual Repick (26.0% vs. 22.0%), and Pattern Lock (56.0% vs. 52.0%).Source: §A.1 · p. 12
- The two B-Unmask variants and Visual PlaceO are lower, while three tasks are unchanged.Source: §A.1 · p. 12
- Together with Table 3, the task distribution supports reuse across distinct closed-loop tasks while indicating that useful reuse requires compatibility with current embodied context.Source: §A.1 · p. 12
A.2 DETAILED LIBERO-PLUS RESULTS
LIBERO-Plus perturbation results show the largest gains for Camera and Noise, while gains are marginal for Robot and Language and FLOWMEM does not lead every column.
A.2 DETAILED LIBERO-PLUS RESULTS
- Camera improves from 80.2% to 88.4% and Noise from 77.5% to 88.3%, the largest perturbation-wise gains.The table reports success rate after suite-specific standard-LIBERO training and zero-shot LIBERO-Plus evaluation.Source: Table 7 · p. 12, §A.2 · p. 13
- Light shows a further 3.7-point gain.Source: §A.2 · p. 13
- Robot and Language gains are only 0.3 and 0.4 points, respectively, and FLOWMEM does not lead every perturbation column.Source: §A.2 · p. 13
- The reported pattern associates useful recomposition with compatible stored trajectories and refinement that aligns the route to the current observation.Source: §A.2 · p. 13
A.3 INFERENCE COST OF LATENT-REASONING REUSE
Under the stated profiling protocol, FLOWMEM improves RoboMME success while reducing mean online control time per episode.
A.3 INFERENCE COST OF LATENT-REASONING REUSE
- The profiling averages 256 episodes per method on an NVIDIA H800 GPU using the same evaluation configuration.Source: §A.3 · p. 13
- For each episode, timing sums observation-to-action-chunk-return time across control decisions before averaging across profiling episodes.Source: §A.3 · p. 13, Table 8 · p. 13
- 15.5 seconds per episode versus 19.6 seconds, while success rises from 46.3% to 48.0%.Table 8 reports mean total online control time from observation to action-chunk return; lower time and higher success rate are better.Source: §A.3 · p. 13, Table 8 · p. 13