Source-linked AI summary
EchoVLA: Robotic Vision-Language-Action Model with Synergistic Declarative Memory for Mobile Manipulation
Min Lin, Xiwen Liang, Bingqian Lin, Jingzhi Liu, Zijian Jiao, Kehan Li, Ziang Yan, Yu Sun, Weijia Liufu, Yuhan Ma, Jiarui Hu, Yuecheng Liu, Shen Zhao, Yuzheng Zhuang, Xiaodan Liang
TL;DR
Existing VLAs largely target short-horizon tabletop tasks and lack structured memory for mobile manipulation. EchoVLA combines scene and episodic memories to guide base-arm diffusion policies, and MoMani provides generated trajectories and real-robot demonstrations. The authors report simulated gains over π0.5 across two task groups and the highest success rate in their real-world comparison.
Problem
Most existing VLAs are restricted to short-horizon tabletop manipulation and rely on current observations, while mobile manipulation requires reasoning over extended tasks and spatial context.
Source: §1 · p. 2Method
EchoVLA combines persistent voxel-map scene memory and time-indexed multimodal episodic memory, retrieving both to guide per-part base-arm diffusion policies.
Source: §1 · pp. 2–3Results
Takeaways & Limitations
MoMani offers an automated pipeline for expert-level trajectories across diverse mobile-manipulation tasks, supplemented by real-robot demonstrations.
Source: §1 · p. 3EchoVLA depends on high-quality depth and pose streams, and cumulative odometry drift can misalign or cause ghosting in Scene Memory.
Source: §6 · p. 15
Abstract
from arXiv · showhide
Recent progress in Vision-Language-Action (VLA) models has enabled embodied agents to interpret multimodal instructions and perform complex tasks. However, existing VLAs are mostly confined to short-horizon, table-top manipulation, lacking the memory and reasoning capability required for mobile manipulation, where agents must coordinate navigation and manipulation under changing spatial contexts. In this work, we present EchoVLA, a memory-aware VLA model for mobile manipulation. EchoVLA incorporates a synergistic declarative memory inspired by the human brain, consisting of a scene memory that maintains a collection of spatial-semantic maps and an episodic memory that stores task-level experiences with multimodal contextual features. The two memories are individually stored, updated, and retrieved based on current observations, task history, and instructions, and their retrieved representations are fused via coarse- and fine-grained attention to guide base-arm diffusion policies. To support large-scale training, we further introduce MoMani, an automated benchmark that generates expert-level trajectories through multimodal large language model (MLLM)-guided planning and feedback-driven refinement, supplemented with real-robot demonstrations. Comprehensive simulated and real-world results demonstrate that EchoVLA substantially improves overall performance, e.g., it achieves the highest success rates of 0.52 on manipulation/navigation tasks and 0.31 on mobile manipulation tasks in simulation, exceeding the strong baseline $\pi_{0.5}$ by +0.20 and +0.11, respectively.
1 Introduction
EchoVLA pairs spatial scene memory with task-level episodic memory to support mobile manipulation, while MoMani supplies expert trajectories and real-robot demonstrations. Simulated and real-world evaluations report gains over strong baselines.
1 Introduction
- Most existing VLAs focus on short-horizon tabletop manipulation and Markovian decisions based only on current observations, limiting extended-task reasoning and long-term spatial understanding.The introduction motivates memory using complementary spatial and experiential memory systems as an analogy for remembering locations and how tasks were accomplished.Source: §1 · p. 2
- EchoVLA combines persistent voxel-map scene memory with time-indexed multimodal episodic memory, retrieving and fusing them through separate coarse- and fine-grained attention for per-part arm/base diffusion control.The design is presented as supporting coherent spatial context and consistent execution over long horizons.Source: §1 · pp. 2–3, Figure 1 · p. 2
- MoMani generates expert-level mobile-manipulation trajectories through MLLM-guided planning and feedback-based refinement, and includes real-robot demonstrations on a holonomic mobile platform.The authors position it as broader in task coverage than existing mobile-manipulation benchmarks and datasets.Source: §1 · p. 3
- +0.20 and +0.11 SR gains over π0.5 [21] on RoboCasa manipulation/navigation and mobile manipulation tasks, respectively; real-world SR reaches 0.44.On TidyBot++ across a 7m × 7m arena, EchoVLA exceeds π0.5 [21] (0.33) and Diffusion Policy (0.32).Source: §1 · p. 3
2 Related Work
Prior VLA work adds diffusion action heads and mobile-base coordination, but the cited approaches lack structured scene-level memory for long-horizon reasoning. Mobile-manipulation benchmarks also vary in task breadth and whether they include base navigation or automated trajectory generation.
2.1 Vision-Language-Action Model
- VLA action-generation approaches include autoregressive action tokens and diffusion- or regression-based heads; AC-DiT [7] models base-manipulator coordination, while π0 [3] and π0.5 [21] address hierarchical trajectory diffusion for mobile manipulation.AnywhereVLA [18] combines classical SLAM with a fine-tuned VLA head for real-world pick-and-place.Source: §2.1 · pp. 3–4
- π0.5 [21] decomposes base and manipulator actions and adds partial episodic memory, but lacks explicit scene-level memory; MemoryVLA [40] uses an implicit perceptual cache rather than explicit spatial maps or task-level experience storage.EchoVLA's scene-plus-episodic memory is presented as addressing these gaps for mobile-manipulation reasoning.Source: §2.1 · p. 4
2.2 Mobile Manipulation Benchmark
- Existing benchmark coverage spans household and kitchen tasks, but several cited suites simplify manipulation, remain kitchen-specific, or omit base navigation; MoMani is described as an automated pipeline for diverse mobile-manipulation trajectories with real-robot demonstrations.The cited examples include RoboCasa's kitchen focus, MoMa-Kitchen's domain specificity, and RoboTwin 2.0's lack of base navigation.Source: §2.2 · p. 4
- Fig. 2 depicts RGB views, point clouds, language, and proprioception encoded as tokens, followed by memory retrieval and per-part diffusion generation of base and arm actions.The section overview places multimodal state representation, memory retrieval, and diffusion-based action generation in the architecture.Source: Figure 2 · p. 5
3 EchoVLA
EchoVLA is a memory-augmented vision–language–action framework for mobile manipulation, combining scene and episodic memory with coarse-to-fine retrieval to condition diffusion-based actions. Its design represents multimodal observations, retrieves spatial and temporal context, and generates separate base and arm actions.
3.1 Problem Formulation
- Visually similar observations can represent different task-progress states, so EchoVLA formulates mobile manipulation as a non-Markovian task requiring memory.At each timestep, the agent observes RGB-D images, proprioception, and an instruction, then outputs continuous arm and base actions.Source: §3.1 · p. 5
3.2 Multimodal State Representation
- A unified token sequence combines language, multi-view image features, depth-derived geometric features, and robot configuration for retrieval and action conditioning.Language and RGB features come from frozen SigLIP towers; depth is encoded by PointAttn, and proprioception by an MLP.Source: §3.2 · p. 6
3.3 Memory Retrieval and Interaction
- Scene memory stores slowly varying spatial structure, while episodic memory retains fine-grained, time-indexed task progress; retrieved features condition the diffusion policy.The model uses a coarse-to-fine hierarchy aligned with the different semantic granularity of the two memories.Source: §3.3 · p. 6
Scene Memory
- Scene memory is an environment-specific voxel structure whose relevant entries are selected within the current local camera frustum.Regions with reconstruction error above 0.5 are updated by exponential moving average with α = 0.2; unchanged regions retain their prior values.Source: Scene Memory · p. 7
Episodic Memory
- Episodic memory is a fixed-size FIFO buffer of recent time-indexed token sequences, preserving interaction details that can distinguish visually similar states.Its window size depends on memory capacity, and it stores original encoded tokens rather than compressed summaries.Source: Episodic Memory · p. 7
Memory Matching and Attention
- The model matches current scene features and multimodal state tokens to stored entries, then applies coarse scene and fine episodic cross-attention.The resulting scene and episodic representations are combined as conditioning input for the diffusion policy.Source: Memory Matching and Attention · pp. 7–8
3.4 Diffusion-based Action Generation
- Independent denoising diffusion processes generate base and arm actions from noisy action samples conditioned on the memory-augmented representation.The paper says this per-part design supports coordinated but decoupled locomotion and manipulation learning, improving scene generalization and cross-task transfer.Source: §3.4 · p. 8
4 MoMani
MoMani is an automated benchmark for mobile manipulation data generation, combining integrated navigation-plus-manipulation trajectories with simulated and real-robot data. Its simulation pipeline generates and filters candidates before selecting and auditing training-ready trajectories.
4 MoMani
- MoMani provides a unified pipeline for integrated navigation-plus-manipulation data and includes both simulated and real-robot sources.The paper distinguishes it from simulation-only platforms and says it delivers expert-level trajectories for complex, multi-stage tasks.Source: §4 · p. 8
4.1 Simulation Data Construction
- A two-stage trajectory-generation pipeline uses sequential target-aligned sampling, safety-aware navigation, and nav-manip stitching; candidates must have zero collisions, ∆pos < 0.05m, ∆ori < 5◦, and 100% task success before ranking by path length and planning cost.A final scene-camera audit ensures the dataset is training-ready.Source: §4.1 · pp. 9–10
4.2 Real-Robot Demonstration Collection
- Real-world demonstrations use a TidyBot++-based holonomic platform, with synchronized RGB-D and stereo cameras recording teleoperated multimodal data at 30 Hz.Successful trajectories are verified by replay, while failed attempts are discarded.Source: §4.2 · p. 10
4.3 Dataset Distribution
- The dataset includes 7,889 simulated episodes, mainly navigation, and 1,200 real-world episodes spanning contact-rich and longer-horizon mobile manipulation tasks.Simulation also includes four mobile-manipulation tasks; the real-world distribution covers six tasks.Source: Figure 4 · p. 9, §4.3 · pp. 10–11
5 Experiment
EchoVLA is evaluated in RoboCasa simulation and real-world mobile manipulation, where it leads the reported baseline comparisons. Ablations examine its observations, memory components, and memory settings, while real-world tests identify dynamic occlusion as a weakness.
5.1 Experimental Setup
- Experiments use RoboCasa and TidyBot++ in a 7m × 7m arena; EnP requires cross-room navigation to retrieve objects.The simulation evaluation reports SR averaged over three seeds and 50 episodes per task, across manipulation, navigation, and mobile manipulation categories.Source: §5.1 · pp. 11–12, §5.2 · p. 12
5.2 Comparison with Baselines in RoboCasa Simulation
- EchoVLA achieves the highest average success rate (0.52) on Manipulation and Navigation, and the highest success rate (0.31) in Mobile Manipulation, where coordinating base and arm control increases complexity.Source: §5.2 · pp. 12–13
- PickPlaceSinkToCounter and TurnOnSinkFaucet show larger performance variation across methods, and EchoVLA performs strongly on both.The paper associates these variations with greater spatial-understanding and precise-manipulation requirements.Source: §5.2 · p. 13
5.3 Ablation Study
- Removing point-cloud input reduces performance in both mobile and non-mobile variants, indicating the importance of geometric depth cues for localization and grasping.The ablation compares RGB-only observations with observations that include point-cloud input.Source: §5.3 · p. 13, Table 3 · p. 13
- Disabling either Episodic Memory or Scene Memory substantially degrades performance; removing RGB or point-cloud signals compounds the effects.The reported ablations cover manipulation and mobile manipulation tasks in RoboCasa.Source: §5.3 · p. 13, Table 3 · p. 13
- L = 8 and τ = 0.5 are selected as optimal; smaller or larger settings respectively risk plan forgetting or latency, and unstable or outdated scene memory.The sensitivity analysis evaluates EM window size L and update threshold τ.Source: Table 4 · p. 14, §5.3 · p. 14
5.4 Real Robot Experiment
- Dynamic occlusion during refrigerator opening degrades EchoVLA’s explicit 3D Scene Memory, while implicit-memory baselines are more robust to those blind spots.The paper attributes the change to rapidly changing geometry as the door opens.Source: §5.4 · p. 14
- 0.44 mean SR on real-robot tasks beats π0.5 (0.33) and Diffusion Policy (0.32).The evaluation reports six basic actions; EnP is the longest-horizon task.Source: §5.4 · p. 14, Table 5 · p. 15
6 Conclusion
The conclusion presents EchoVLA as a dual-memory framework for mobile manipulation and MoMani as a benchmark spanning simulated and real-robot evaluation. It reports strong baseline comparisons while noting depth, pose, odometry, and cold-start constraints.
6 Conclusion
- EchoVLA combines 3D Scene Memory and Episodic Memory to address non-Markovian decisions and mitigate perceptual noise.Source: §6 · p. 15
- MoMani spans procedurally generated simulation tasks and real-robot evaluations.Source: §6 · p. 15
- Performance depends on high-quality depth and pose streams; cumulative odometry drift can misalign or ghost Scene Memory, and novel environments begin with sparse 3D memory.The paper proposes loop-closure or visual-SLAM and active exploration or generative 3D priors as future directions.Source: §6 · p. 15