PaperlayerView exact revision on arXiv

Source-linked AI summary

Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control

Qinghang Liu, Conglin Wang, Zongsheng Li, Boyong He, Han Gao, Pengcheng Wang, Tao Sun, Lige Liu, Chang Liu

arXiv:2610.29329v1cs.RO

TL;DR

Long-horizon manipulation can depend on historical information outside a robot’s short observation window, while memory-augmented policies still search and reason over stored history at execution time. PAL organizes streaming observations into hierarchical memory and writes future-relevant subtasks before execution. It reports higher success than the strongest baseline on RoboMME and higher overall success than two baselines across four real-world tasks.

  • Problem

    Memory-dependent manipulation can require information from earlier observations, while existing memory-augmented policies still search stored history and infer the next subtask during execution.

    Source: §1 · p. 1

  • Method

    PAL uses a visual buffer, semantic-event memory, and executable subtask list, with a two-stage training pipeline combining dense update supervision and verified self-labeling.

    Source: §1 · p. 2

  • Results

    56.05% RoboMME success versus 44.51% for FrameSamp-Modul; real-world overall success was 60.83% across four tasks, versus MemER at 45.00% and FrameSamp-Modul at 28.33%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments on RoboMME and four real-world tasks show PAL improves memory-dependent robot control, particularly when relevant evidence must be retained over long horizons.

    Source: §6 · p. 9

    PAL’s sequential executable subtask list cannot encode conditional branches during execution.

    Source: §6 · p. 9

Abstract

from arXiv · show

Long-horizon robot manipulation requires a robot to remember task-relevant information beyond its short observation window. Existing memory-augmented policies typically retrieve relevant information from history and reason over it at execution time, placing a heavy reasoning burden on the high-level model. To address this challenge, we propose PAL (\textbf{P}lan first, \textbf{A}ct \textbf{L}ater), a hierarchical memory framework for vision-language robot manipulation. It maintains a short-term visual buffer, a semantic-event memory, and an executable \texttt{subtask\_list}. A high-level vision-language model (VLM) converts streaming visual observations into the \texttt{subtask\_list}. At execution time, the VLM selects the currently relevant subtask from the \texttt{subtask\_list} and passes it to a low-level vision-language-action (VLA) policy. To teach the model what to write to memory and when to update it, we introduce a two-stage data generation procedure that first expands sparse update supervision and then uses self-labeling with task-grounded verification to generate training data that better match the state distribution encountered during execution. We evaluate our approach on memory-dependent benchmarks and real-world manipulation tasks. On RoboMME, PAL achieves a \textbf{56.05\%} success rate, outperforming the strongest baseline, FrameSamp-Modul, at 44.51\%. In real-world experiments across four manipulation tasks, \pal{} reaches \textbf{60.83\%} overall success, surpassing MemER (45.00\%) and FrameSamp-Modul (28.33\%). These results show that proactively writing future-relevant subtasks into hierarchical memory improves long-horizon visual reasoning and robot control. Code and videos can be found on our website: https://qinghangliu.github.io/pal-webpage/

1 INTRODUCTION

Long-horizon manipulation can require historical information that is unavailable in current observations, while memory-bank approaches still leave the high-level model to search and infer at execution time. PAL writes future-relevant subtasks into hierarchical memory before they are needed and evaluates this approach on benchmark and real-world tasks.

1 INTRODUCTION

  • Memory-dependent tasks can require recalling earlier progress or an object’s previously observed location or state, beyond a short window of recent frames.For example, repetitive manipulation requires remembering progress, while exploration can require retrieving information about an object found earlier.Source: §1 · p. 1
  • Memory banks avoid repeatedly processing raw history, but the high-level model must still search stored entries and infer the next subtask.The introduction also notes that redundant bank contents can make retrieval difficult.Source: §1 · pp. 1–2
  • When evidence for a future goal appears before its execution, PAL can store that subtask and retrieve it after transient visual evidence disappears.The introduction illustrates this with storing a cube-pickup subtask while a highlight is still visible.Source: §1 · p. 2
  • PAL uses a short-term visual buffer, semantic-event memory, and executable subtask list, updated by a high-level VLM that selects a subtask for the low-level VLA policy.Streaming updates compress visual history, and stored subtasks can be retrieved without searching a large memory bank.Source: §1 · p. 2, Figure 1 · p. 2
  • 56.05% RoboMME success versus 44.51% for FrameSamp-Modul; real-world overall success was 60.83% across four tasks, versus MemER at 45.00% and FrameSamp-Modul at 28.33%.The authors report these results as evidence of gains in simulated and real-world memory-dependent manipulation.Source: §1 · p. 2
  • PAL’s training pipeline combines dense update-timing supervision with self-labeling and a task-grounded verifier to better match training data to execution rollouts.The contribution summary describes the pipeline as learning reliable hierarchical-memory updates.Source: §1 · pp. 2–3

2 RELATED WORKS

Prior work addresses long-horizon robot control through hierarchical interfaces and memory-augmented VLA systems, but the latter still require execution-time search over stored history. PAL instead records future-relevant subtasks when supporting evidence appears and retains them in an editable list.

Vision-language-action models for robot control

  • Many VLA policies use current or recent observations, motivating memory mechanisms for tasks that require information from earlier history.The passage describes progress in VLA generalization through larger-scale training, broader robot data, and continued experience-driven learning.Source: §2 · p. 3

Hierarchical planning and subgoal-conditioned control

  • Hierarchical interfaces separate high-level subtask selection from low-level action generation, but do not specify how earlier observations inform decisions.τ0-VLA adds execution memory and world-model-guided comparison of candidate outcomes.Source: §2 · p. 3

Memory-augmented VLA models and benchmarks

  • Most memory-augmented VLA methods store and retrieve historical information, but execution-time search costs grow with stored history; PAL records future-relevant subtasks early and retains them in an editable list.MaP-WAM also plans ahead, but the passage distinguishes it as storing only the next step.Source: §2 · p. 3

3 PRELIMINARIES

The setup considers tasks where correct action may depend on information missing from the current observation, making full-history policies challenging as visual context grows. PAL organizes memory across recent images, recorded events, and executable subtasks, with the high-level model selecting a subtask for low-level execution.

Problem setup

  • Memory-dependent tasks may require information not visible in the current observation, while a full-history policy faces growing visual context containing redundant frames.The setup denotes the instruction, visual observation, low-level action, and raw interaction history as L, o_t, a_t, and H_t.Source: §3 · p. 3

Hierarchical memory formulation

  • PAL’s hierarchical memory has a short-term visual buffer for recent RGB images, a semantic-event list for relevant events that occurred, and an executable list with one subtask per entry.The high-level VLM updates either list and selects the current subtask to pass to the low-level policy.Source: §3 · pp. 3–4

Plan first, Act later

  • PAL separates observing evidence for a subtask from executing it, allowing a subtask written earlier to be selected for later execution.The subtask list is built around storing a subtask when visual evidence shows it will be needed before its execution time.Source: §3 · p. 4

4 METHODOLOGY

PAL separates writing future-relevant subtasks from executing them, then uses streaming memory updates and two-stage training to teach the model what to store and when. The methodology covers both memory construction and querying, and training the model to update memory and select a subtask.

4.1 OVERVIEW

  • A subtask can be written to memory at t1 and selected for execution later at t2, separating when evidence reveals a goal from when it is acted on.Source: §4.1 · p. 4
  • The method separates memory construction and querying during VLM rollouts and inference from training the VLM to update memory and select the current subtask.Source: §4.1 · p. 4

4.2 STREAM-BASED MEMORY REASONING

  • The subtask list grows by appending new entries; event memory is cleared when new subtasks are added and otherwise retained, while predicted events are appended.Source: §4.2 · pp. 4–5
  • The visual buffer appends new observation chunks between memory updates and is cleared after an update to limit evidence accumulation and ease subsequent reasoning.Source: §4.2 · p. 5

4.3 TWO-STAGE TRAINING WITH SELF-LABELING AND VERIFICATION

  • Stage 1 expands each single-timestep update annotation into samples across query times, assigning empty targets before presumed sufficient evidence and reference labels later.The samples use visual buffers containing observations accumulated since the preceding update.Source: §4.3 · pp. 5–6
  • Stage 2 rolls out the Stage 1 model and uses task-grounded verification to reject incorrect or duplicate updates, retaining trajectories accepted at every required update point.The verifier checks predictions against reference labels and whether the memory already stores the entry.Source: §4.3 · p. 6

5 EXPERIMENTS

The experiments evaluate PAL on simulated RoboMME tasks, training ablations, and four real-world manipulation tasks. PAL outperforms the strongest baseline in both simulated and real-world results, while the ablations examine how update timing affects memory quality.

5.1 SIMULATED EXPERIMENTS

  • PAL’s simulated evaluation uses RoboMME task groups covering different conditioning-video lengths, in-episode adaptation, and episode-start list construction.The ablation compares variants that share the hierarchical memory architecture and low-level VLA policy, differing in training procedure and inference-time update gate.Source: §5.1 · pp. 6–7, Table 1 · p. 7, §5.2 · p. 8
  • 56.05% overall success on RoboMME exceeds FrameSamp-Modul’s 44.51% by 11.54 percentage points; PAL also leads on 4 of 16 tasks.The comparison focuses on methods with hierarchical high-level-to-low-level control: PAL also exceeds MemER (42.38%) and GPT-6-astra + VLA (39.75%).Source: §5.1 · p. 7, Table 1 · p. 7

5.2 ABLATION STUDY

  • 56.05% overall success for PAL compares with 43.39% for Sparse Label and 37.35% for Stage 1 Only.The update gate greatly improves final-list accuracy over Stage 1 Only by providing approximate update timing, separating when to update from what to update.Source: §5.2 · p. 8, Table 2 · p. 8
  • Stage 1 Only updates too often, while Sparse Label under-updates and can fail to update any subtask list on Long-video tasks.PAL’s update frequency is closer to ground truth and yields higher final-list accuracy; Stage 2 regularizes the overly frequent updates learned in Stage 1.Source: §5.2 · p. 8, Table 2 · p. 8

5.3 REAL-WORLD EXPERIMENTS

  • The real-world evaluation uses four tasks, with 30 evaluations per task across difficulty levels and three repeats of each 10-episode evaluation.The tasks target progress tracking, visual target recall, long-video recall, and reasoning over a temporally dense action sequence.Source: §5.3 · pp. 8–9, Figure 4 · p. 9
  • 60.83% overall real-world success is PAL’s highest result among the compared methods, with its largest gains on longer-horizon tasks.MemER scores 0.00% on PickXtimes versus PAL’s 53.33%, while FrameSamp-Modul scores 13.33% on VideoPlaceOrder.Source: §5.3 · p. 9, Table 3 · p. 9

6 CONCLUSION

PAL converts streaming observations into semantic events and future-relevant subtasks before they are needed for control, using a two-stage training procedure to learn memory updates. The conclusion reports improved memory-dependent control, especially over long horizons, while noting that its sequential executable memory cannot represent conditional branches.

6 CONCLUSION

  • PAL incrementally converts streaming visual observations into semantic events and a subtask list before they are needed for control.Its training combines dense update supervision with verified self-labeling to learn memory updates under the model’s rollout distribution.Source: §6 · p. 9
  • Experiments on RoboMME and four real-world manipulation tasks show improved memory-dependent robot control, particularly when relevant evidence must be retained over long horizons.Source: §6 · p. 9
  • PAL’s sequential subtask list cannot encode conditional branches during execution.The authors propose branchable executable memory as a future extension.Source: §6 · pp. 9–10

A ALGORITHMS

  • The appendix provides pseudocode for streaming updates to visual, event, and executable memories, and for Stage 2 self-labeling with task-grounded verification.Source: A ALGORITHMS · p. 12

B PAL TRAINING DETAILS

PAL’s Stage 2 self-labeling builds on the Stage 1 model and uses verification to produce training episodes. The procedure uses one pass when Stage 1 rollouts are reliable and cyclic relabeling when they are weaker.

B.2 STAGE 2 SELF-LABELING

  • Stage 2 starts from the Stage 1 LoRA adapter and retains its optimizer, batch size, and LoRA configuration.The Stage 2 mode depends on how reliable the Stage 1 model already is.Source: §B.2 · p. 13
  • The verifier requires exact matches for non-coordinate text and coordinate predictions within an L2 distance strictly below 10 pixels.Without coordinates, the exact-text criterion alone applies.Source: §B.2 · p. 13
  • One-pass self-labeling is used when Stage 1 already produces reliable rollouts; accepted episodes are retained for training alongside original supervised labels for tasks not self-labeled.The reported single pass accepts most episodes across its self-labeled tasks, so no further rounds are needed.Source: §B.2 · pp. 13–14
  • In cyclic self-labeling, accepted episodes rise from 31 of 200 in the first round to 170 in the final round across 15 rounds.The run relabels PatternLock and VideoPlaceOrder after successive 200-step training blocks; Figure 5 tracks accepted episodes and normal rollout successes.Source: §B.2 · p. 14, Figure 5 · p. 15

C.1 FULL SIMULATION RESULTS

PAL performs best on several RoboMME tasks, leads MemER on BinFill, and remains substantially weaker than the strongest baseline on some tasks.

C.1 FULL SIMULATION RESULTS

  • PAL achieves 95.20% on VideoUnmask, 90.80% on ButtonUnmask, 80.00% on VideoPlaceButton, and 83.20% on VideoPlaceOrder, performing best on each task.Its margins over the strongest baseline are 13.87, 18.80, 20.00, and 39.20 percentage points, respectively.Source: §C.1 · p. 15
  • PAL reaches 65.20% on BinFill, compared with 56.67% for MemER.Source: §C.1 · p. 15
  • PAL is weaker than the strongest baseline on RouteStick (22.40% vs. 66.67%) and SwingXtimes (79.60% vs. 92.00%).Source: §C.1 · p. 15
  • PAL scores 4.80% on InsertPeg and 5.20% on StopCube, where the best alternatives score 11.20% and 46.00%, respectively.VideoRepick and both swap tasks also remain difficult; their best scores are 32.00%, 38.00%, and 21.33%, respectively.Source: §C.1 · p. 15

C.2 DATASET CONSTRUCTION

The appendix describes sparse-label construction and the evaluation setup, then analyzes baseline failure modes and PAL’s remaining control limitations.

C.2 DATASET CONSTRUCTION

  • Sparse labels mark only event-memory or executable-subtask-list changes, with simulator oracle annotations used to identify what happened and when its information becomes available.Episode-start tasks receive a full list at the first query; changing-environment tasks add events and subtasks as evidence becomes sufficient; StopCube’s list is the current subtask.Source: §C.2 · pp. 15–16

C.3 IMPLEMENTATION DETAILS AND BASELINES

  • PAL is evaluated on 50 episodes per task per seed, with conditioning videos streamed to the robot despite being available in full at the start.The appendix says this setting evaluates real-time reasoning; the cited baseline results come from RoboMME and use nine runs per method.Source: §C.3 · p. 16

C.4.1 GPT-6-ASTRA + VLA

  • GPT-6-astra + VLA struggles with subtle visual changes, including whether a button press succeeded, and performs worse on ButtonUnmask and ButtonUnmaskSwap than on their video counterparts.Source: §C.4.1 · p. 17

C.4.2 PAL

  • PAL’s poor results on InsertPeg and StopCube involve precise insertion and rapid reaction to a moving object, requirements outside the work’s main focus on memory updates and long-horizon reasoning.On RouteStick, reducing action-chunk size from 16 to 8 raised success from 22.4% to 46.0%.Source: §C.4.2 · p. 17

D.1 DATA COLLECTION AND PROCESSING

The appendix describes how trajectories are annotated and converted into action-aligned data, then specifies real-world task settings and robot hardware.

D.1 DATA COLLECTION AND PROCESSING

  • Coarse action and subtask boundaries come from gripper actions and stationary intervals, then GPT-5.6-Sol refines them into event and subtask labels.The fine-grained annotations are based on the coarse annotations and recorded trajectory videos.Source: §D.1 · p. 17
  • Processing removes stationary portions, preserves frames around transitions and task boundaries, synchronizes observations with states and actions, and rewrites each retained action to target the next observation.Both camera views are cropped and resized with fixed camera-specific regions before trajectories are re-indexed.Source: §D.1 · p. 17

D.2 SETTINGS AND IMPLEMENTATION DETAILS

  • Real-world difficulty varies with the task-relevant history to retain; VideoUnmask has no difficulty levels.PickXtimes, VideoPlaceOrder, and PatternLock use different history-dependent task configurations.Source: §D.2 · p. 18, Table 7 · p. 18
  • The real-world setup uses a Franka Research 3 with front and wrist cameras, an RTX 4090, and a low-level controller running at approximately 950 Hz.Excluding network latency, the VLA generates commands at approximately 10–12 Hz.Source: §D.2 · p. 18

D.3 EVALUATION PROTOCOL

The protocol defines task-specific success and failure conditions, repeats each episode three times, and presents representative PAL rollouts across four tasks.

D.3 EVALUATION PROTOCOL

  • Each episode is repeated three times, and runs exceeding 1,300 timesteps or triggering the robot safety lock count as failures.VideoPlaceOrder, PatternLock, and VideoUnmask use pre-recorded conditioning videos; there are 30 evaluations per task.Source: §D.3 · p. 18
  • PickXtimes succeeds only after the specified number of lifts followed by 100 timesteps with the arm raised; extra contact or a cycle exceeding 650 timesteps fails.The arm and box start from fixed positions in every episode.Source: §D.3 · p. 18
  • VideoPlaceOrder succeeds when the released box overlaps the designated target, while release outside all targets or contact with an incorrect target fails.The box starts at the workspace center and the arm at a fixed pose.Source: §D.3 · p. 18
  • PatternLock succeeds when the gripper visits grid points in the prescribed order and fails if it leaves the grid or crosses an incorrect point.A point counts as visited when any part of the gripper passes directly above it.Source: §D.3 · p. 18

D.4 ROLLOUT VISUALIZATION

  • Figure 6 shows eight sampled frames per rollout, with PatternLock, PickXtimes, VideoPlaceOrder, and VideoUnmask arranged from top to bottom.The accompanying text describes the demonstrated sequence or target for each task.Source: Figure 6 · p. 19, §D.4 · p. 20

D.3 EVALUATION PROTOCOL

  • VideoUnmask succeeds when the robot points to the correct box after the demonstration and fails when it points to the wrong box.The arm starts from a fixed pose.Source: §D.3 · p. 19

D.5 MORE ANALYSIS

The analysis identifies baseline failures in progress tracking and target grounding. The prompt examples show how PAL updates a future subtask list during video conditioning and selects a current subtask during execution.

D.5 MORE ANALYSIS

  • 0.00% success: MemER repeatedly outputs PickXtimes’ initial subtask instead of tracking completed cycles; FrameSamp-Modul’s 30.00% VideoUnmask success is near the 33.33% random-choice rate.Observed FrameSamp-Modul rollouts repeatedly point to the middle box rather than the requested cube.Source: §D.5 · p. 20

E PROMPT

  • The two inference examples cover VideoPlaceOrder list updates from conditioning video and PatternLock selection of the current subtask during execution.Each query includes a fixed output schema, task, current memory, selected frames, and the model response.Source: E PROMPT · p. 20
  • In the video-phase example, conditioning images update the ordered subtask list, while the prompt asks for a newly completed event rather than a predicted next event.The example concerns placing a red cube on the second target used in the demonstration.Source: E PROMPT · p. 20
  • The four conditioning frames are at timesteps 349, 378, 407, and 436; the four execution frames are at 58, 71, 83, and 96.Both sets are shown in timestep order.Source: Figure 7 · p. 21, Figure 8 · p. 21

Related papers

Loading 2610.29329v1…