PACE: Stage-Consistent Long-Horizon Robot Manipulation via Progress-Aligned Context for Execution

I. INTRODUCTION

• Existing methods may require predefined decompositions or manual annotations, or use history without continually revising which demonstration segment guides behavior.PACE is evaluated on compositional simulation and real-world robotic tasks with held-out demonstrations, alongside stage-level analyses of mitigation.Source: §I · pp. 1–2
• PACE uses training-only stage-boundary supervision and causal, episode-local fast-weight memory to align demonstration context with realized execution progress, without inference-time stage labels or stage-specific policies.The memory encodes realized action–observation transitions; the unified diffusion action expert uses the resulting context for action generation.Source: §I · p. 2
• 33.3% to 73.3%: PACE raises mean success on two-step Block Routing tasks, where cross-stage observation aliasing is pronounced.It also improves success from 88.9% to 94.0% on LIBERO-Gen Goal Chain and from 79.1% to 83.3% on Spatial Combination.Source: §I · p. 2
• Stage confusion arises from cross-stage observation aliasing and differences in the speed at which demonstrators and robots complete behavioral stages.Visually similar observations can require different future actions depending on completed progress.Source: §I · pp. 1–2, Figure 1 · p. 1

A. Demonstration-Conditioned Robot Policies

• Demonstration-conditioned policies use demonstrations as prompts, but generally do not continually reinterpret them according to realized progress, leaving them susceptible to cross-stage observation aliasing.BPP treats a complete perception–action demonstration as a behavioral prompt for closed-loop action generation [4].Source: A. Demonstration-Conditioned Robot… · p. 2

B. Temporal Alignment and Stage-Structured Manipulation

• Temporal alignment and hierarchical approaches can rely on ambiguous local matches, fixed offline alignments, predefined decompositions, semantic labels, or separate skill policies.PACE instead learns ordered demonstration structure without manual stage labels and dynamically interprets it during execution.Source: B. Temporal Alignment and… · p. 2

C. History and Memory for Long-Horizon Manipulation

• History-aware policies retain execution history, but PACE uses an episode-local fast-weight state to modulate observation queries before prompt cross-attention, letting history select relevant demonstration content.Prior approaches primarily use history to improve action prediction.Source: C. History and Memory for Long-Horizon… · p. 2

III. PROBLEM STATEMENT

• Demonstration observations and actions are indexed by timestep j, and T_p denotes the demonstration length.The demonstration and execution follow the same high-level operation order, while trajectory length and stage duration may differ.Source: §III · pp. 2–3
• At each timestep, the policy predicts an action sequence from the current observation and strictly causal episode history; the demonstration is the sole behavior specification, with no test-time stage label or progress signal.Policy parameters remain fixed at test time, and episode-local state resets between episodes.Source: §III · pp. 2–3
• The latent correspondence ϕ maps execution timesteps to demonstration positions monotonically, but need not advance linearly or at a constant rate.The relevant demonstration guidance may span a contiguous interval around ϕ(t).Source: §III · p. 3

A. Architecture

• PACE combines a demonstration prompt encoder, dual-edge attention supervision, a stateful progress-alignment decoder, and a unified diffusion action expert.The decoder and expert infer execution progress from realized history and generate closed-loop actions.Source: A. Architecture · p. 3

B. Demonstration Prompt Encoder

• Each 20-step demonstration chunk fuses visual, proprioceptive, and action features into one token using attention pooling and a temporal positional embedding.The prompt retains the first visual frame per chunk while preserving its proprioceptive and action sequences.Source: B. Demonstration Prompt Encoder · p. 3, Figure 3 · p. 4
• The resulting demonstration memory is a continuous, ordered sequence at fixed temporal resolution rather than a sequence of discrete event labels.Figure 3 depicts each chunk as an ordered prompt token.Source: B. Demonstration Prompt Encoder · p. 4, Figure 3 · p. 4

C. Dual-Edge Attention Supervision

• Boundary detection combines changes in action spectrum, motion direction, mean, end-effector speed, pausing, jerk, and gripper state, then filters weighted signals and adds the trajectory start as an anchor.Continuous motion channels are robustly normalized, while gripper transitions serve as explicit boundary cues.Source: C. Dual-Edge Attention Supervision · p. 4
• Dynamic programming monotonically aligns execution and demonstration anchors while accounting for temporal position, boundary type, repeated matches, and skipped prompt anchors.The alignment preserves behavior order while tolerating a small number of missing or redundant detections.Source: C. Dual-Edge Attention Supervision · p. 4
• Aligned left and right stage boundaries form masked Gaussian targets over prompt chunks that supervise aggregated and per-query attention; supervision is omitted for an open final stage without a reliable right boundary.The equally weighted boundary distributions describe a stage interval, allowing flexible within-stage aggregation and accommodating speed variation.Source: C. Dual-Edge Attention Supervision · p. 4

D. Stateful Progress-Alignment Decoder

• PACE stores all weights and biases of a fixed-size two-layer MLP, updating its episode-local state after each completed 12-control-step execution segment.Each segment encodes realized action–observation transitions from valid actions and start/end multi-camera features; the state stores no growing trajectory buffer.Source: D. Stateful Progress-Alignment Decoder · p. 4
• Fast-memory writes use only preceding completed segments during training and only realized transitions during deployment, keeping the state update causal.The reconstruction objective defines the state transition, while training backpropagates through at most 16 consecutive writes.Source: D. Stateful Progress-Alignment Decoder · pp. 4–5
• Each decoder layer injects a gated memory readout into observation tokens before they query the complete demonstration, so the same observation can retrieve different stages under different histories.The resulting context is concatenated with the original observation tokens to condition action generation.Source: D. Stateful Progress-Alignment Decoder · p. 5

E. Unified Diffusion Action Expert

• PACE uses a single conditional 1D U-Net across tasks and behavior stages, with task semantics and progress specified by the observation and progress-aligned demonstration context.The shared expert is trained with diffusion denoising, dual-edge attention supervision, and effective-support regularization.Source: E. Unified Diffusion Action Expert · p. 5
• Figure 4 presents representative execution sequences for simulation Spatial Combination, Goal Chain, and Block Routing, plus real-world Block Routing.The simulations cover three task families, and the real-robot Block Routing evaluation includes visual perturbations, action errors, and stochastic interaction outcomes.Source: Figure 4 · p. 5
• At inference, dual-edge supervision is removed and no explicit event segmentation or discrete stage prediction is required; PACE updates memory from current observations and realized history, then replans after new outcomes.It executes part of the diffusion expert's action sequence before replanning.Source: E. Unified Diffusion Action Expert · p. 5

A. Experimental Setup

• Simulation covers Goal Chain, Spatial Combination, and Block Routing, probing unseen goal compositions, unseen spatial relations, and recurring observations across stages.Block Routing includes three single-region routes and nine ordered two-region routes.Source: A. Experimental Setup · p. 5
• Real-robot Block Routing tests behavioral consistency across simulation and reality under visual perturbations, action errors, and stochastic interaction outcomes.Source: A. Experimental Setup · p. 6
• BPP is the principal baseline, with π0.5 also evaluated in simulation; methods share the training split, checkpoint-selection protocol, test demonstrations, success criteria, and rollout protocol.Source: A. Experimental Setup · p. 6
• Success requires every operation stage to finish in the correct order, and evaluation provides no test-time stage label, progress signal, or semantic subgoal.The primary metric is mean episode-level task success across test episodes.Source: A. Experimental Setup · p. 6

B. Simulation Results

• PACE reaches 83.3% on Spatial Combination and 94.0% on Goal Chain; on Block Routing it exceeds BPP by 40.0 and 34.5 points on 2-step and 4-step tasks.On unseen compositions, PACE exceeds π0.5 by 3.4 points on Spatial Combination and 14.8 points on Goal Chain; on Block Routing, the gains are 60.0 and 37.6 points on 2-step and 4-step tasks.Source: B. Simulation Results · p. 6
• PACE uses neither test-time stage labels nor stage-specific action experts, so its gains come with no additional task-semantic supervision.Source: B. Simulation Results · p. 6

C. Real-Robot Results

• PACE achieves 90.0% success on real-robot 2-step tasks versus 40.0% for BPP, and 43.3% versus 3.44% on 4-step tasks.The real-robot setting has greater variation in action duration, contact error, and visual appearance than simulation, often changing how long a stage takes.Source: C. Real-Robot Results · p. 6

VI. ABLATIONS

• The ablation section asks whether gains reflect reduced stage confusion, whether both stage structure and stateful alignment are needed, and whether attention tracks execution progress.Source: §VI · p. 6

A. Does PACE Reduce Stage Confusion?

• Wrong-stage failures fall from 14 in BPP (46.7% of trials) to one in PACE (3.3%), with the authors attributing most of the 56.6-point success-rate gain to fast-weight memory.The task repeatedly revisits the same region; each method receives 30 evaluation rollouts.Source: A. Does PACE Reduce Stage Confusion? · p. 6

B. Role of Stage Structure and Stateful Progress Alignment

• Neither effective-support regularization nor dual-edge attention supervision improves success alone; the authors conclude stage supervision must interact with execution state to revise demonstration retrieval.Source: B. Role of Stage Structure and… · p. 6

C. Does Demonstration Attention Track Execution Progress?

• BPP attention is diffuse and nearly stationary, while full PACE follows an approximately monotonic trajectory aligned with execution progress; without FWM, attention switches between stages under aliased observations.Source: C. Does Demonstration Attention Track… · p. 6
• With the current observation fixed, replacing the fast-state history with the later-stage history shifts attention toward that later prompt stage, while resetting the state shifts it toward the earlier stage.Matched histories concentrate attention within the corresponding stage boundaries; shuffled history changes the distribution even when its attention center remains similar in one example.Source: C. Does Demonstration Attention Track… · p. 7

• PACE represents the complete demonstration as an ordered multimodal prompt, learns stage structure with training-only dual-edge attention supervision, and uses causal episode-local memory to align it with execution progress.Source: §VII · pp. 7–8
• Fig. 8 compares attention under visually aliased first and second To block occurrences and under matched, earlier, later, and shuffled histories.Source: Figure 8 · p. 8
• PACE improves complete-task success on compositional simulation tasks and Block Routing and substantially reduces wrong-stage failures.Source: §VII · p. 8
• Evaluation covers few task families and rollouts; monotonic stage correspondence, action-change cues, fixed-duration chunks, and fast-weight overhead constrain applicability.Source: §VII · p. 8