Imagine the Future, Internalize the Gist: Efficient VLA Reasoning via Internalized Spatiotemporal Imagination

1 INTRODUCTION

• VLA models use multimodal observations and language instructions to produce actions, and Chain-of-Thought reasoning is one approach to decomposing complex instructions and adapting plans to feedback.The introduction notes that effective end-to-end mapping requires extensive, high-quality demonstration data.Source: §1 · p. 1
• Existing VLA reasoning primarily plans from observed states without explicitly anticipating scene evolution, while repeated full spatiotemporal rollouts at each inference step add computation.These are the two limitations the framework is designed to address.Source: §1 · p. 2
• Latent Spatiotemporal Reasoning imagines task-relevant future scene evolution in visual representation space to guide actions without costly pixel-level video generation.The paper contrasts this with language-based reasoning, which may lose fine-grained spatial information and lead to imprecise actions.Source: §1 · pp. 1–2, Figure 1 · p. 2
• Scene Gist Memory internalizes reasoning-derived scene–behavior associations in a compact Scene Gist Token for reuse, bypassing explicit future imagination and full reasoning rollouts at inference.The paper describes this as analogous to repeated experience consolidating familiar behaviors into procedural memory.Source: §1 · p. 2
• IG-VLA’s gist policy reaches 169.5 ms per action chunk on a single NVIDIA A6000, with 6.38× speedup over FastWAM-IDM (1,081 ms).It also achieves 1.78× and 1.55× speedups over FastWAM (1-step) (302 ms) and ImageWAM (263 ms), respectively.Source: §1 · p. 3
• On LIBERO-Plus’s Language suite, IG-VLA achieves an 85.6% success rate, outperforming the strongest baseline by nearly 6%.The reported experiments span LIBERO, LIBERO-Plus, and VLABench.Source: §1 · pp. 2–3

2 RELATED WORK

• OpenVLA uses end-to-end policy learning, π0 scales policy learning across diverse tasks, HAMSTER strengthens spatial reasoning, and TraceVLA adds 2D visual trajectories.These examples illustrate distinct ways foundation VLA models connect perception and action.Source: §2 · p. 3
• VLA reasoning methods include prompting with supervised traces, data-centric reasoning-action demonstrations, visual subgoals, and reinforcement learning with verifiable rewards.The passage names EcoT, RAD, CoT-VLA, and GRPO-based approaches, respectively.Source: §2 · p. 3
• The related-work discussion notes that current reasoning methods can require costly reasoning supervision and sophisticated reward engineering.Source: §2 · p. 3

3 METHODOLOGY

• IG-VLA combines future-trajectory reasoning with Scene Gist Tokens in a VLA framework built on a conventional architecture.Source: §3 · p. 3, §3.1 · p. 3

3.1 MODEL ARCHITECTURE

• The model encodes visual observations and language instructions, predicts future scene representations, and uses them with current context to guide future actions.Source: §3.1 · p. 4
• A gist compressor summarizes the reasoning-enhanced representation, while a gist encoder predicts that target from current context for the gist policy.Source: §3.1 · pp. 4–5

3.2 MODEL TRAINING WITH FUTURE REASONING AND SCENE GIST MEMORY

• Training proceeds by first learning the reasoning policy, then training Scene Gist modules with the pretrained reasoning policy frozen.Source: §3.2 · p. 5

3.2.1 LEARNING SPATIOTEMPORAL FUTURE REASONING FOR TRAJECTORY ALIGNMENT

• The spatiotemporal module predicts visual transitions from current observations and instructions; transition supervision focuses on task-relevant changes rather than static scene content.The objective also encourages directional consistency between predicted and target transitions.Source: §3.2.1 · p. 5

3.2.2 INTERNALIZING SPATIOTEMPORAL REASONING INTO SCENE GIST MEMORY

• The gist compressor is trained to match full-reasoning-policy actions, then the frozen compressor supplies targets for an encoder trained with representation alignment and action consistency.Source: §3.2.2 · pp. 5–6
• At inference, the gist encoder predicts one Scene Gist from current context and conditions action generation, bypassing the reasoning rollout, implicit action reasoner, and compressor.Source: §3.1 · p. 5, §3.2.2 · p. 6

4.1 EXPERIMENTAL SETUP

• Experiments use LIBERO (Liu et al., 2023), LIBERO-Plus, and five VLABench tracks, with comparisons against representative VLA and world-model baselines.The VLABench reasoning-module comparison uses identical training data; inference latency is evaluated on one NVIDIA A6000 GPU.Source: §4.1 · p. 6

4.2 SIMULATION EXPERIMENTS

• On LIBERO, both policies reach 98.9% average success; LIBERO-Plus Language improves from 79.7% to 85.6%, and VLABench Intention Score reaches 66.68.On VLABench, average Progress Score is 46.11 versus ACoT-VLA's 47.40; the reported LIBERO-Plus average is 88.5%.Source: §4.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
• 169.5 ms per action chunk gives the gist policy 1.78× and 1.55× speedups over FastWAM (1-step) and ImageWAM, respectively.Scene Gist reduces overall GPU time from 420.7 ms to 169.5 ms; the gist policy retains a 98.9% LIBERO average success rate versus ACoT-VLA's 98.5%.Source: §4.2 · p. 8, Table 4 · p. 8

4.3 ABLATION STUDY

• A 10-keyframe horizon yields 98.90% LIBERO average success, compared with 98.10% at three keyframes and 97.80% at 15.The authors attribute the longer-horizon drop to accumulated prediction errors and redundant or unreliable future information, and use K = 10 by default.Source: §4.3 · p. 8, Table 5 · p. 8
• With λreg = 1, λdir = 0.1 gives 99.4% success on LIBERO-Spatial; increasing λdir to 0.5 or 0.8 drops success to 37.0% or 25.7%.The authors use λdir = 0.1 as a lightweight auxiliary constraint in the main experiments.Source: §4.3 · p. 8, Figure 5 · p. 9
• A two-layer gist-policy MLP reaches 98.9% average LIBERO success, versus 97.8% for one layer; three layers slightly reduce success to 98.8%.Training cost rises from 3.45 GPU hours at two layers to 5.79 GPU hours at three, so the authors select two layers.Source: Figure 6 · p. 9, §4.3 · p. 9

4.4 REAL-WORLD DEPLOYMENT

• Real-world evaluation uses a dual-arm UR3 for screwdriver pick-and-place and compares baseline success rates over 16 trials per method.Source: Figure 7 · p. 9

• IG-VLA targets VLA reasoning's lack of explicit future-scene anticipation and the high inference cost of repeated reasoning rollouts.Source: §5 · p. 9
• Latent spatiotemporal reasoning provides task-relevant future guidance for action generation, while Scene Gist Memory internalizes reasoning knowledge for efficient inference.Source: §5 · p. 9
• Experiments on LIBERO, LIBERO-Plus, and VLABench report improved manipulation and generalization, and real-world robotic experiments validate the framework beyond simulation.Source: §5 · p. 9

• LLMs were used solely as a general-purpose writing assistant.Source: AI USE STATEMENT · p. 10
• Their role was limited to correcting grammar and fixing typographical errors.Source: AI USE STATEMENT · p. 10
• They also polished the language for clarity and readability.Source: AI USE STATEMENT · p. 10

• During training, the model encodes current and future observations, predicts future latent visual transitions from multimodal context, and uses anticipated representations to inform actions.Source: §A.1 · p. 14
• Scene Gist Memory compresses reasoning-enhanced representations into a gist target, then trains a lightweight encoder with representation- and action-level supervision to predict it from current context.Source: §A.1 · p. 14
• At inference, the gist encoder retrieves a Scene Gist from the current observation and instruction, conditioning action-chunk generation while bypassing the reasoning module and future-state rollout.Source: §A.1 · pp. 14–15

A.2 MORE DETAILS ABOUT REAL-WORLD DEPLOYMENT

• The deployment uses two 6-DoF UR3 arms and three RGB cameras, but only the right arm performs the screwdriver pick-and-place task.The resulting state/action space is 14-dimensional; policies are fine-tuned from pi05_base on 77 teleoperated demonstrations at 15 Hz.Source: §A.2 · p. 15
• Predicted delta-EEF action chunks are converted server-side to joint targets using forward kinematics and damped-least-squares inverse kinematics before execution.Each 10-step chunk is executed at 5 Hz, with a joint-velocity cap of 2.5 rad/s.Source: §A.2 · p. 15
• Policies are compared over 16 trials each using randomized screwdriver placements from the same distribution, with binary task-success rate reported.All three policies share the fine-tuning configuration and differ only in method.Source: §A.2 · p. 15