Chronos Policy: Learning Robot Policies on Physical Time

1 INTRODUCTION

• Fixed-rate observation–action sequences use sequence position as a proxy for physical time, obscuring elapsed intervals when sensing or inference timing varies.For a moving object, identical image displacements can imply different velocities when images are 20 ms versus 200 ms apart.Source: §1 · p. 1
• Chronos Policy conditions on observation capture times and requested action execution times, using time-aware RoPE to represent elapsed time in attention without changing token content.It represents observation-to-observation, observation-to-action, and action-to-action timing.Source: §1 · pp. 2–3
• Training randomizes which observations are selected and when action chunks begin, while timestamps let the policy interpret those varied schedules and retain measurements fixed-rate downsampling would discard.The training variation is intended to expose the policy to timing schedules that may occur during deployment.Source: §1 · p. 2
• Across dynamic, multisensory, and quasi-static tasks, Chronos Policy improves robustness to temporal mismatch while preserving nominal performance on standard tasks.Dynamic manipulation depends on observation-to-action timing, while multisensory manipulation depends on observation-to-observation timing.Source: §1 · pp. 2–3

Asynchronous multisensory learning

• Multisensory methods preserve or fuse asynchronous sensor streams using separate modules, frequency-aware embeddings, cross-attention, or modality-specific policies.These approaches establish the importance of preserving high-frequency feedback.Source: §2 · p. 3

Inference delay and chunk continuity

• Prior approaches address inference delay through action consistency, delay compensation, or faster inference; Chronos Policy makes remaining sensing and execution timing explicit, complementing latency reduction.The cited methods include overlapping inference and execution, delay-compensated targets, and variable-noise diffusion schedules.Source: §2 · p. 3

Explicit time in robot control

• Earlier control policies use time to adapt behavior speed or duration, whereas Chronos Policy represents when observations are captured and actions executed.The passage describes this formulation as covering observation rate, sensor delay, cross-modal timing, and action execution delay in one policy.Source: §2 · p. 3

Continuous-time representations and attention

• Continuous-time attention and rotary time representations have appeared in irregular time series and video; Chronos Policy applies this view to robot imitation learning with heterogeneous observation and action timestamps.The related work includes mTAN, ContiFormer, variable-frame-rate video models, VideoRoPE, and BulletTime.Source: §2 · pp. 3–4

3 METHOD

• The policy conditions a diffusion transformer on observation capture times and desired action execution times, keeping the timing interface consistent from training through deployment.Each available observation retains its capture time, while noisy action tokens receive queried execution times.Source: §3 · p. 4, §3.4 · p. 6

3.1 POLICY FORMULATION ON PHYSICAL TIME

• Observation-time spacing represents sensing rate, observation-to-action gaps represent measurement delay and execution latency, and each modality can retain its own capture times.This avoids aligning multimodal measurements to a common sampling grid.Source: §3.1 · p. 4

3.2 ENCODING PHYSICAL TIME IN THE POLICY

• Time-aware RoPE keeps token embeddings unchanged and uses physical timestamps as positional coordinates, making attention depend on elapsed time and remain invariant to a shared shift in clock origin.Image, proprioceptive, force/torque, and action tokens use their respective capture or queried execution times.Source: §3.2 · p. 5

3.3 TRAINING WITH TEMPORAL RANDOMIZATION

• Training randomizes observation rates, modality delivery delays, and action-chunk start times while preserving capture timestamps, exposing the policy to deployment-like timing schedules.The sampled measurements can include data that fixed-rate downsampling would otherwise discard; explicit timestamps disambiguate histories whose token positions span different intervals.Source: §3.3 · p. 5

3.4 USING PHYSICAL TIME AT DEPLOYMENT

• At deployment, action chunks are executed asynchronously; already-past queried actions are discarded and the remaining actions follow their requested execution times.Simulation sets the initial action delay from injected inference latency; on the real robot it is estimated from previously completed requests.Source: §3.4 · p. 6

4.1 TASKS AND EVALUATION SETUP

• The evaluation covers dynamic Conveyor, multisensory Pivoting, and six quasi-static tasks under nominal, observation-timing, inference-latency, and combined conditions.Pivoting combines vision, proprioception, and force/torque measurements acquired at different rates and times.Source: §4.1 · p. 6, §4 · p. 6

4.2 WHEN IS TIME INFORMATION USEFUL?

• Under inference delays up to 200 ms on Conveyor, success rises from 1.8% with Diffusion Policy to 76.4% with Chronos Policy.Under nominal timing, success increases from 73.1% to 82.0%; with combined perturbations, Chronos reaches 32.7% while Diffusion Policy fails entirely.Source: §4.2 · p. 6
• On Pivoting, success under inference delays up to 200 ms rises from 51.3% to 60.2%, and under combined timing changes from 19.6% to 38.4%.The advantage of explicit time grows as temporal mismatch increases.Source: §4.2 · p. 6

4.3 WHAT MAKES A POLICY TIME-AWARE?

• Temporal randomization improves Conveyor robustness alone, but on Pivoting it lowers perturbed-condition average success from 48.4% to 21.6% without timestamps.With explicit time, randomization raises perturbed-condition averages from 48.8% to 63.1% on Conveyor and from 33.3% to 55.3% on Pivoting.Source: §4.3 · pp. 7–8
• Chronos-RoPE beats Chronos-AddTime in every non-nominal Conveyor setting and every Pivoting condition, raising temporal-condition averages from 51.5% to 63.1% and 33.5% to 55.3%, respectively.Chronos-AddTime is slightly better on Conveyor under nominal timing; the authors hypothesize RoPE benefits from separating token content and time and directly modulating attention with elapsed time.Source: §4.3 · p. 8

4.4 DOES TIME HURT STANDARD QUASI-STATIC MANIPULATION?

• Across six quasi-static tasks, nominal performance is comparable, while average success across five perturbed conditions improves from 59.7% with Diffusion Policy to 73.4% with Chronos Policy.The largest gains occur under wide sensing variation and combined perturbations.Source: §4.4 · p. 8

4.5 DOES TIME AWARENESS IMPROVE REAL-WORLD ROBUSTNESS?

• Chronos Policy achieves the highest task score in every evaluated real-world condition across Dynamic Cup, Peg Insertion, and Pivoting.The advantage persists under observation, inference, and combined timing perturbations.Source: §4.5 · p. 9, Figure 5 · p. 9

• Chronos Policy conditions on when observations are captured and actions should execute, providing a unified representation of sensing rate, measurement delay, and inference latency.Source: §5 · p. 9
• Across dynamic, multisensory, and standard quasi-static manipulation in simulation and on real robots, Chronos Policy improves robustness to temporal mismatch while preserving nominal performance.Source: §5 · p. 9
• Future work would extend Chronos policies beyond fixed action rates to variable control frequencies and temporal scaling, so behavior can run faster or slower than demonstration.Source: §5 · p. 9

A.1 POLICY ARCHITECTURE AND INPUT REPRESENTATIONS

• The shared backbone is a 16-layer diffusion transformer with width 512 and eight attention heads; non-image tokens use self-attention and image patches use cross-attention.Proprioceptive and action vectors are separate tokens, with force/torque tokens added for Pivoting and Insertion.Source: §A.1 · p. 13, Figure 6 · p. 13

A.2 BASELINES AND TIME-AWARE VARIANTS

• Time-aware RoPE uses observation capture times and queried action execution times, while the Diffusion Policy baseline uses sequence position without timestamps.The baseline’s temporal-randomization variant receives randomized schedules and action-start delays but not their physical times.Source: §A.2 · p. 13
• Chronos AddTime maps timestamp Fourier features through an MLP and adds the embedding to timed tokens; Chronos-RoPE instead leaves token embeddings unchanged and encodes time in attention.The two variants share data, token layout, diffusion conditioning, and temporal randomization.Source: §A.2 · p. 13
• Real-Time Chunking constrains each next action chunk with previously committed actions while inference overlaps execution; RTC and Chronos estimate inference latency from completed requests.Diffusion Policy and RTC do not receive observation capture timestamps as policy inputs.Source: §A.2 · pp. 13–14

A.3 TRAINING DETAILS

• Policies are trained for 50 epochs with AdamW, a 3 × 10−4 learning rate, and an encoder learning rate one tenth as large.Training also uses ℓ1 action loss, BF16, and cosine schedules for diffusion noise and learning rate.Source: §A.3 · p. 14

B.1 CHECKPOINT SELECTION AND EVALUATION SETUP

• Checkpoints are selected by nominal validation success every five epochs, and results average three training seeds.Conveyor and Pivoting ablations use 450 episodes per method and condition.Source: §B.1 · p. 14

B.2 TEMPORAL SETTINGS

• Training samples observation rates of 12.5–33.33 Hz and delivery delays of 0–50 ms; Pivoting force/torque uses 1–100 Hz and 0–100 ms delays.Half of action chunks begin with zero delay; the rest sample a start delay up to 100 ms.Source: §B.2 · p. 14
• Evaluation varies inference delay up to 100 or 200 ms, with Combined using the large-observation setting and the large inference-delay distribution.Source: §B.2 · p. 14

• On Conveyor, randomization alone raises success from 15.6% to 60.9% under Small observation variation and from 14.2% to 47.3% under delays up to 100 ms, but Wide sensing improves only to 6.4%.Nominal success falls from 73.1% to 53.6%, and both variants remain near zero under Combined timing changes.Source: §B.3 · p. 15
• On Pivoting, randomization alone lowers success in every condition, including nominal performance from 61.6% to 29.8%, where hidden timing creates ambiguity across vision, proprioception, and force/torque.Source: §B.3 · p. 15
• On Conveyor, RoPE without randomization reaches 76.4% and 69.3% under 100 and 200 ms delays, but only 20.2% under Wide sensing and 10.7% under Combined.Adding randomization raises Wide and Combined results to 45.6% and 32.7%; Pivoting shows the same pattern, with RoPE alone at 9.1% and 4.2% versus 48.0% and 38.4% for the full model.Source: §B.3 · p. 15
• Timestamps make each sensing and execution schedule interpretable, while randomization exposes the policy to a broader range of schedules; their combination is more robust than either alone.Source: §B.3 · p. 15
• RoPE outperforms AddTime in every perturbed condition: on Conveyor, success is 45.6% versus 24.0% under Wide sensing and 32.7% versus 7.8% under Combined perturbations.On Pivoting, RoPE reaches 48.0% versus 23.6% under Wide sensing and 38.4% versus 16.7% under Combined; AddTime has higher Conveyor nominal success, 85.8% versus 82.0%.Source: §B.3 · p. 15

B.4 STANDARD MANIPULATION: TASK-LEVEL RESULTS

• The clearest gains occur on Assembly, Coffee, and Pouring, especially under observation and combined timing changes.Source: §B.4 · p. 15
• Chronos Policy achieves comparable nominal performance and better performance under temporal perturbations across the six standard manipulation tasks.The results are reported as success rate (%) and mean ± sample SD over three training seeds, with 50 episodes per condition per seed.Source: §B.4 · p. 15, Table 4 · p. 16
• Table 4 reports full results for the six tasks, with Diffusion Policy (DP) as the transformer-based baseline.Source: Table 4 · p. 16

C.1 TASK SETUP AND DEMONSTRATIONS

• The three real-world tasks are Dynamic Cup, Peg Insertion, and Pivoting, scored as full, partial, or failed success; Peg Insertion and Pivoting each include multiple configurations.Dynamic Cup varies saucer speed, position, and rotation direction; Peg Insertion tests two hole orientations, while Pivoting tests four object or configuration settings.Source: §C.1 · p. 17, Table 5 · p. 17
• All tasks use two RGB observations and histories of relative pose and gripper width; Peg Insertion and Pivoting additionally use force/torque histories from each finger.The force/torque streams are recorded at approximately 360 Hz and sampled at an approximately 90 Hz history rate.Source: §C.1 · p. 17

C.2 TEMPORAL SETTINGS AND EXECUTION

• Evaluation includes inference delays sampled from 0, 50, 100, 150, or 200 ms, while Combined applies both observation perturbation and inference delay.Observation streams have independent delivery delays; force/torque timing applies to Peg Insertion and Pivoting.Source: §C.2 · p. 17, Table 6 · p. 17

C.3 FULL RESULTS AND ANALYSIS

• Task score is 100(F + 0.5P)/N, where F and P are full and partial successes across N trials.Source: §C.3 · p. 17
• Dynamic Cup results are grouped by motion direction, with five trials per direction and an overall total of 10 trials per policy and timing condition.Source: Table 7 · p. 18
• Peg Insertion results are grouped by forward and backward hole orientation, with five trials per orientation and 10 trials overall.Source: Table 8 · p. 18
• Pivoting covers large and small in-distribution erasers and out-of-distribution box and sideways settings, with five trials per scenario and 20 overall.Source: Table 9 · p. 18