PaperlayerView exact revision on arXiv

Source-linked AI summary

Physical Time as a Missing Dimension of Robot Policy Memory Dynamics

Yushen Liang, Yue Peng, Baosheng Jin, Tianluo Zhang, Xinyu Zhang, Shuyi Zhou, Zhuoran Chen, Xinqi Liu, Shenji Wan

arXiv:2610.20524v1cs.RO

TL;DR

Robot-policy memories driven by observation sequences can treat updates as equal steps even when identical histories span different physical durations and call for different actions. TFP evolves a persistent memory over measured intervals and uses it to condition diffusion and flow-matching policies. It achieves higher average success than the compared memory alternatives, with evidence from physical-robot timing and manipulation benchmarks.

  • Problem

    Observation-sequence-driven memories can treat each update as the same amount of physical time, although identical histories may span different durations and require different actions.

    Source: §1 · p. 1

  • Method

    TFP predicts memory content and time scales from observations and prior memory, advances memory over measured intervals, and uses the updated state to condition diffusion and flow-matching policies.

    Source: §1 · p. 2

  • Results

    Equal-weight average success across tasks and schedules was 77.5% for TFP with Diffusion Policy versus 62.1% for S4D+∆t, and 73.3% with π0.5 versus 66.8% for S4D+∆t.

    Source: §4.1 · p. 6

  • Takeaways & Limitations

    Memory readouts recover elapsed event time and motion information after occlusion, while clock-scaling interventions advance or delay actions in timing tasks.

    Source: §5 · p. 9

Abstract

from arXiv · show

Existing robot policy memories typically evolve with observation sequences, implicitly treating each update as the same amount of physical time. Yet identical observation histories can span different durations and require different actions, exposing physical time as a missing dimension of policy memory. We introduce Temporal Flow Policy (TFP), which learns memory dynamics to retain past events and evolve the memory state over measured time intervals. Observations and prior memory determine the content and time scales of each transition; elapsed time determines how far the state advances. TFP conditions diffusion and flow-matching policies through this memory state and is trained with their native imitation objectives. We evaluate TFP with Diffusion Policy and $\pi_{0.5}$ through controlled comparisons of memory designs. These experiments show that successful timing depends on both measured intervals and retained history. TFP achieves higher average success than the recurrent and state-space memories we equip with the same intervals and than a variant that feeds the interval to a learned gate. Memory readouts recover the time since an event and motion information after occlusion. Changing the supplied intervals systematically delays or advances actions, linking time inputs to behavior. Real-robot tasks requiring timing or memory, together with LIBERO evaluations, demonstrate benefits in physical and general manipulation. These results suggest viewing robot memory not as stored observation history, but as a state that retains past events and evolves with physical time.

1 INTRODUCTION

Observation sequences can span different physical durations and require different actions, so policy memory must account for elapsed time. TFP uses measured intervals to evolve memory and guide action, and controlled evaluations report higher average success than specified memory alternatives.

1 INTRODUCTION

  • Identical observation histories can span different physical durations and require different actions, a mismatch that can arise with varying observation rates or inference latency.The introduction illustrates this with tea steeping and inferring a moving cup’s velocity across different observation intervals.Source: §1 · p. 1, Figure 1 · p. 2
  • TFP retains past events in a memory state that evolves over measured time intervals and conditions diffusion and flow-matching policies.Its memory dynamics and conditioning are jointly learned from causal demonstration histories using each backbone’s native imitation objective, without event-time labels.Source: §1 · p. 2
  • Controlled comparisons with Diffusion Policy and π0.5 show TFP has higher average success than recurrent and state-space memories given the same intervals and a learned-gate alternative.The evaluation includes real-robot timing and sequencing tasks and LIBERO manipulation tasks.Source: §1 · p. 2

2 RELATED WORK

Prior robot-policy work uses history, timestamps, sensing schedules, or time-dependent control conditions, while continuous-time models offer ways to evolve state over intervals. The related work identifies how elapsed physical time should shape memory updates as an open question.

2 RELATED WORK

  • Robot-policy memories retain earlier cues and subtasks through compact tokens, recurrent queries, memory banks, or short-term visual and long-term language memory.The described systems encode temporal context or evaluate timing behavior, but do not specify how elapsed physical time should shape memory evolution.Source: §2 · pp. 2–3
  • Other robot approaches expose time through timestamped history, sensor-rate-specific buffers, or remaining-time and clock-progression conditions.The related work leaves open whether elapsed time should enter memory updates as an input feature or as an interval of state evolution.Source: §2 · p. 3
  • Continuous-time recurrent models, state-space models, and gated memory pathways provide mechanisms for time-dependent state evolution, while TFP combines measured intervals with event-dependent memory dynamics for action generation.The passage names LTC, CfC, S4D, Mamba, and M-CfC as examples of these model families.Source: §2 · p. 3

3 METHOD

TFP summarizes timed history in a persistent memory state, evolves that state over measured intervals, and uses it to condition an action generator. Its memory dynamics and conditioning are trained from causal demonstration histories with the backbone’s native imitation objective.

3.1 MEMORY FOR CONTROL IN PHYSICAL TIME

  • At each query, TFP combines a persistent memory state summarizing timed history with the backbone’s current observation and task context to generate an action chunk.The memory is reset at episode boundaries, and elapsed time is computed from successive observation timestamps.Source: §3.1 · p. 3

3.2 MEMORY DYNAMICS IN PHYSICAL TIME

  • Observation features and prior memory predict candidate content and channel-wise time constants, while the measured interval sets how far memory advances toward the candidate.Separate heads let TFP learn what memory represents and how quickly it changes; the transition is an exact solution of a local relaxation process.Source: §3.2 · pp. 3–4, Figure 2 · p. 4
  • With predicted coefficients fixed, the transition preserves the state at zero duration, approaches the candidate monotonically, and composes across intervals.Different time constants create multiple temporal responses, and adaptive rates support rapid updates around events and slower evolution during waiting.Source: §3.2 · p. 4

3.3 ACTING ON MEMORY

  • The memory state augments the backbone’s native decoder condition through adaptive normalization, alongside its current context.Memory updates once per query and remains fixed during solver steps; persistent-state storage is O(dh) per episode, independent of episode length.Source: §3.3 · pp. 4–5

3.4 LEARNING MEMORY DYNAMICS FROM DEMONSTRATIONS

  • The backbone’s native imitation objective jointly trains memory dynamics and the conditioning projection from demonstrated actions using causal histories and measured intervals.Controlled timing models reconstruct memory from causal prefixes, while LIBERO training uses consecutive episode segments with gradients truncated between segments.Source: §3.4 · p. 5

4 EXPERIMENTS

Experiments test whether measured intervals and retained history support control across observation schedules, timing interventions, real-robot tasks, and LIBERO evaluations. TFP outperforms the reported reactive baselines on real-robot manipulation and LIBERO, while controlled tests examine timing, memory readouts, and design choices.

4.1 DOES POLICY MEMORY NEED PHYSICAL TIME?

  • The evaluation varies observation schedules while holding task durations and target speeds fixed, using Button Press, Passage, and Interception to test duration, event age, and motion estimation.Models are evaluated at 10 Hz, 5 Hz, and under jitter and dropout; trials are paired across methods within each backbone.Source: §4.1 · pp. 5–6
  • Reactive policies fail Button Press under every schedule, while short history achieves at most 4.7% on Passage, where recurrent memory must retain the cue throughout the wait.Short history improves Interception.Source: §4.1 · p. 6
  • Averaged equally across tasks and schedules, TFP achieves 77.5% with Diffusion Policy versus 62.1% for S4D+∆t, and 73.3% with π0.5 versus 66.8%.Passage performance remains high across schedules, supporting responses to remembered cues despite changes in observation cadence.Source: §4.1 · p. 6, Table 1 · p. 6

4.2 HOW MEMORY CONTROLS BEHAVIOR

  • With weights and physical query schedules fixed, halving supplied intervals delays button release and passage entry, while doubling them advances those actions; both scalings eliminate success on these tasks.Each directional shift occurs in all 50 paired trials for π0.5 Button Press and for both backbones on Passage.Source: §4.2 · pp. 6–7, Figure 4 · p. 7
  • A linear probe predicts time since the Passage cue from TFP memory with 0.04 s mean absolute error, versus about 0.83 s using episode time alone; memory also retains speed information after occlusion.The result supports event-relative timing beyond episode progress.Source: §4.2 · p. 7
  • Removing history from the candidate, using static time constants, or replacing interval-based evolution with a learned ∆t gate each lowers mean success relative to Full TFP.On Diffusion Policy Button and Passage, the learned gate achieves 14.5% and 15.0%, versus 81.2% and 97.3% for TFP’s exponential update.Source: §4.2 · p. 7, Table 2 · p. 7
  • During Delayed Passage, channels show distinct temporal responses: some adapt rapidly around events, while others maintain slower traces during waiting.The visual aligns observations, write gains, and channel time constants on a physical time axis.Source: §4.2 · p. 8, Figure 5 · p. 8

4.3 DOES TFP IMPROVE REAL-ROBOT CONTROL?

  • TFP improves all five real-robot tasks, with mean gains of 47 and 50 percentage points for Diffusion Policy and π0.5; LIBERO success is 98.7% versus 96.9% for the reported reactive baseline.On LIBERO-plus, success rises from 91.4% to 93.7%.Source: §4.3 · p. 8, Table 3 · p. 8, §4.4 · p. 9

5 CONCLUSION

The conclusion frames physical time as a missing dimension of observation-driven policy memory and presents TFP as a memory state that evolves over measured intervals. It links this design to improved manipulation results and argues that memory dynamics should follow physical time when actions depend on event age.

5 CONCLUSION

  • TFP retains past events in a memory state whose content and timescales depend on observations and history, while measured intervals determine how far it advances.The state conditions existing diffusion and flow-matching action generators.Source: §5 · p. 9
  • Comparisons with recurrent and state-space memories supplied with the same intervals support using elapsed time to govern memory evolution.Readouts recover event age and motion information after occlusion; clock scaling advances or delays actions, and resets and conditioning ablations support a role for accumulated history.Source: §5 · p. 9

A ADDITIONAL METHOD DETAILS

The additional method details specify sequence training, memory initialization, and the interval-dependent relaxation underlying TFP. They also describe the controlled timing protocols and computational costs used to evaluate it.

A.1 SEQUENCE TRAINING

  • Training reconstructs memory from causal prefixes; LIBERO carries hidden state across consecutive segments while truncating gradients between segments.For controlled π0.5 timing tasks, the recurrence starts at zero and gradients flow through the full prefix to four supervised queries.Source: §A.1 · p. 12

A.2 MEMORY INITIALIZATION AND PARAMETERIZATION

  • Memory starts at zero at episode boundaries, with a 0.1-second bootstrap interval before subsequent timestamp-difference updates.Diffusion Policy and π0.5 use different channel counts and time-constant parameterizations.Source: §A.2 · p. 12

A.3 RELATION TO CANONICAL LTC DYNAMICS

  • TFP shares a local relaxation form with canonical LTC but predicts candidate content and time constants independently; its input- and state-dependent dynamics also relate to CfC.For recurrent baselines, measured duration sets TFP’s integration interval rather than serving only as an added feature.Source: §A.3 · p. 13, §C.1 · p. 14

B PROPERTIES OF THE MEMORY UPDATE

  • With candidate and rate fixed over a query interval, TFP’s memory moves toward the candidate, stays unchanged at zero duration, and composes consistently across split intervals.The update recomputes candidate and rate at each query; its structural constraints cover nonnegative intervals, including those outside the training range.Source: B PROPERTIES OF THE MEMORY UPDATE · p. 13, §B.1 · p. 13

B.2 DURATION SENSITIVITY

  • Duration changes the state when it differs from the candidate, while sensitivity vanishes at equilibrium.The predicted rate controls how quickly the difference decays.Source: §B.2 · p. 13

C.1 RECURRENT AND CONTINUOUS-TIME MODELS

  • The evaluation compares recurrent and state-space memories, including S4D, while noting that Mamba’s learned discretization step is not a measured physical interval.The comparison tests elapsed-time-aware alternatives within the same action-conditioning setup.Source: §C.1 · p. 14, §C.2 · p. 14

C.2 STATE-SPACE MODELS AND COMPUTATIONAL COST

  • Per query, candidate and time-constant heads cost O(dh(dx + dh)), the elementwise update costs O(dh), and persistent state uses O(dh) storage per episode.Memory projection costs O(dhdc); dx and dc are observation-feature and decoder-conditioning widths.Source: §C.2 · p. 14

D.1 CONTROLLED TIMING PROTOCOLS

  • Controlled timing evaluations use button, passage, and interception tasks under fixed, jittered, or dropout observation schedules, with models trained at 10 Hz.The protocols specify physical success criteria and compare Diffusion Policy and π0.5 configurations, including recurrent and state-space baselines.Source: §D.1 · pp. 14–15, Table 5 · p. 15

D.2 MEMORY-STRUCTURE ABLATIONS

The interventions vary memory structure, accumulated history, update rules, or supplied intervals while controlling other aspects of the comparison. Clock scaling changes event timing in the predicted direction across paired trials.

D.2 MEMORY-STRUCTURE ABLATIONS

  • Memory-structure ablations compare static time constants and a learned interval gate while holding observation features, memory width, and action-conditioning pathways fixed.The learned gate receives observation, prior memory, and the measured interval; the cited table caption identifies the ablation results as success rates across schedules.Source: §D.2 · p. 15, Table 6 · p. 16

D.3 ACCUMULATED MEMORY AND ACTION CONDITIONING

  • Accumulated-memory ablations remove memory conditioning or reset memory at every observation, while retaining current-observation and diffusion-timestep conditioning.The reset variant keeps the memory update and measured interval but removes information carried across observations.Source: §D.3 · p. 16

D.4 UPDATE RULE

  • Update-rule comparisons test a shared scalar time constant and rational retention against the full update across four schedules and three seeds.Rational retention keeps measured time and candidate updating but replaces exponential retention.Source: §D.4 · p. 16

D.5 FROZEN-WEIGHT INTERVENTIONS

  • Halving supplied intervals lengthened π0.5 button contact in all 50 paired trials, while doubling them shortened it; both backbones entered the passage later or earlier, respectively, in all 50 trials.The interventions kept trained weights and physical task schedules fixed while changing supplied intervals or stored memory.Source: §D.5 · p. 16

D.6 TEMPORAL AND MOTION INFORMATION IN MEMORY

This section describes probes of frozen policy representations for event timing and motion information. It specifies the tested representations, targets, and evaluation splits, without reporting probe outcomes in these passages.

D.6 TEMPORAL AND MOTION INFORMATION IN MEMORY

  • Diffusion Policy probes test whether frozen observations, memory, or their concatenation encode button-contact duration and time since a disappeared passage cue.The targets supervise only the probe; elapsed time since episode start is also tested as a feature.Source: §D.6 · p. 16, Table 8 · p. 17
  • π0.5 readouts compare time-since-cue predictions from memory variants augmented with elapsed episode time against the episode clock alone.Table 9 reports additional readout comparisons using frozen policy representations.Source: §D.6 · p. 16, Table 9 · p. 17
  • Interception probes evaluate signed velocity, speed magnitude, and direction from frozen representations after target occlusion and before robot movement.The split groups all direction–speed variants from each scene together; single-frame controls use encoded images and robot state.Source: §D.6 · pp. 17–18
  • Figure 5 visualizes a demonstration with a cue at 1.7 seconds, showing selected memory channels and heatmap rows ordered by median time constant.The displayed layout is selected by its readout error being closest to the median among analyzed layouts.Source: §D.6 · p. 18

D.7 EVENT-ALIGNED MEMORY IN THE OTHER SIMULATION TASKS

The visualizations align memory dynamics with physical-time events in simulation and LIBERO. The manipulation-variant setup and training costs are also documented.

D.7 EVENT-ALIGNED MEMORY IN THE OTHER SIMULATION TASKS

  • The Figure 6 and 7 heatmaps show write gains for all 128 channels on a fixed [0, 1] scale, alongside selected raw-memory channels.Channels are ordered by their median time constant over each displayed sequence; the Passage panel shows time constants.Source: §D.7 · p. 18
  • During Timed Button Press, channel 106 rises during contact and drops around release; during occlusion, channels 1 and 115 evolve while the robot stays still.The button-press contact interval is 3.09–6.33 s; in the occlusion sequence the target moves at 0.16 m/s and becomes invisible at 0.35 s.Source: Figure 6 · p. 18, Figure 7 · p. 18

D.8 MANIPULATION MEMORY ABLATIONS

  • The reproduced π0.5 manipulation variants share demonstrations, action horizon, optimizer, execution settings, and evaluation protocol.The largest π0.5 runs used over 100 GB GPU memory and approximately 80 hours on four H200 GPUs; storage scales with active episodes and sequence-processing cost depends on training unroll length.Source: §D.8 · p. 19

D.9 MEMORY DYNAMICS IN LIBERO

  • In a successful LIBERO rollout, write gains change around grasping, release, and door interaction, and depend on measured intervals and predicted time constants.The figure shows observations, channel-wise gains, and gain summaries over policy queries.Source: Figure 8 · p. 19, §D.9 · p. 19

D.10 REAL-WORLD SETUP AND EVALUATION PROTOCOL

The real-robot evaluation uses a Galaxea A1 setup and compares TFP-equipped Diffusion Policy experiments with standard reactive policies. Training and trial protocols are specified, including timing criteria for tea steeping and staged heating.

D.10 REAL-WORLD SETUP AND EVALUATION PROTOCOL

  • A single-arm Galaxea A1 uses a parallel gripper, wrist-mounted fisheye camera, and static Intel RealSense D435 camera.The real-robot Diffusion Policy uses the compact visual implementation and the same backbone, TFP memory update, and action-conditioning interface as controlled simulations.Source: §D.10 · p. 19
  • Each task uses 50 training demonstrations and 20 evaluation trials per method; both backbones train at fixed 0.1-second query intervals.The baseline for each backbone is its standard reactive policy without persistent memory.Source: §D.10 · p. 19
  • Tea steeping requires 3.0 ± 0.5 s of immersion, while staged heating requires both preheating and cooking intervals to last 3.0±0.5 s in prescribed order.Event timestamps are annotated in evaluation videos, with the same timing criteria applied to all methods.Source: §D.10 · p. 19

E.1 CONTROLLED TIMING: VARIATION ACROSS TRAINING SEEDS

The appendix reports seed-level variability for controlled-timing and LIBERO evaluations, with aggregation and rollout uncertainty described. The supplied LTC+∆t results give task-average success across three seeds.

E.1 CONTROLLED TIMING: VARIATION ACROSS TRAINING SEEDS

  • Controlled-timing task averages first weight 10 Hz, 5 Hz, jitter, and dropout equally within each seed, then aggregate three seed-level averages.Overall success in the main text additionally weights the three tasks equally, regardless of rollout counts; Table 10’s continuation reports schedule results and rollout 95% confidence intervals.Source: §E.1 · p. 20, Table 10 · p. 20
  • 19.7±2.3% for Diffusion Policy and 23.6±2.4% for π0.5 are LTC+∆t overall success, averaged equally across tasks within each seed.Table 10 reports task-average success as mean ± sample SD over three independent training seeds.Source: Table 10 · p. 20, §E.1 · p. 20

E.2 LIBERO SEED VARIABILITY AND ROLLOUT UNCERTAINTY

  • Table 11 reports LIBERO and LIBERO-plus success as mean ± sample SD across three training seeds, with suite or perturbation conditions equally weighted in Avg.Each standard LIBERO suite has 10 tasks and 50 rollouts per task and seed for evaluated variants; the table gives TFP suite-level rollout 95% confidence intervals.Source: Table 11 · p. 21, §E.2 · p. 21

F TASK SEQUENCES

The task-sequence figures show simulation and real-robot examples, with frames ordered left to right. The Interception expert sequence depicts the target becoming hidden before the robot begins moving.

F TASK SEQUENCES

  • In the Interception expert sequence, the target becomes invisible to the policy before the robot starts moving; dashed outlines show its hidden true position for visualization only.The dashed target is absent from policy inputs.Source: Figure 9 · p. 22, F TASK SEQUENCES · p. 22
  • The simulation and real-robot task-sequence frames show action order from left to right, and real-robot columns pair Agent and Wrist camera views.The real-robot examples include Water pouring, Tea steeping, and Staged heating.Source: Figure 9 · pp. 22–23

Related papers

Loading 2610.20524v1…