Source-linked AI summary
AdaTempo: Learning Shared Relative Tempo from Demonstrations for Faster Robot Manipulation
Jiale Cao, Yike Niu, Zhengrong Xue, Huazhe Xu
TL;DR
Imitation-trained visuomotor policies inherit slow demonstration timing, but uniform acceleration can mishandle phases with different acceleration tolerance. AdaTempo aligns demonstrations’ relative-tempo patterns and uses their consensus to resample training data, achieving up to 3.57× speedup with a stronger success–speed trade-off than representative baselines.
Problem
Uniform acceleration is unreliable because manipulation phases tolerate acceleration differently and can lose important states or demand untrackable motions when shortened.
Source: §I · p. 1Method
AdaTempo aligns demonstrations by phase, aggregates their relative-tempo profiles into a consensus, and converts it into a continuous speedup profile for offline resampling before standard policy training.
Source: §III · p. 2, §I · p. 2Results
Across 11 tasks, AdaTempo reaches up to 3.57× SOD, with 75.5% success at 2.83× mean simulation SOD and 77.8% real-world success versus ACT’s 58.3%.
Source: B. Simulation Results · p. 6, §V · p. 7Takeaways & Limitations
Cross-demonstration relative tempo is a useful self-supervised cue for phase-dependent speedup.
Source: §V · p. 8Reported evaluations cover five simulated and six real-world tasks.
Source: §V · p. 7
Abstract
from arXiv · showhide
Visuomotor policies trained via imitation learning often inherit the unnecessarily slow timing of teleoperated demonstrations. Yet uniform speedup is unreliable because different phases of a manipulation task tolerate acceleration differently. In this work, we introduce AdaTempo, a self-supervised method that accelerates visuomotor policies by exploiting shared relative-tempo structure in demonstrations. AdaTempo establishes phase correspondence, aggregates the aligned relative tempo into a consensus, and maps it to a continuous speedup profile used to resample demonstrations into accelerated training trajectories. Training standard policies such as ACT or Diffusion Policy on these resampled trajectories directly embeds the desired tempo in the learned behavior, without runtime tempo selection or online retiming. Extensive evaluations show that AdaTempo achieves up to a $3.57\times$ speedup and yields a stronger success--speed trade-off than the original policies and representative acceleration baselines.
I. INTRODUCTION
AdaTempo uses shared relative-tempo patterns across demonstrations to set phase-dependent acceleration and train policies on resampled trajectories. The paper reports up to 3.57× speedup and a stronger success–speed trade-off than the original policies and representative acceleration baselines.
I. INTRODUCTION
- Different manipulation phases tolerate acceleration differently, so uniform speedup can skip important states or request motions controllers cannot track.Effective acceleration must determine both its magnitude and how it is distributed over task progress.Source: §I · p. 1
- Existing acceleration approaches often rely on policy-derived proxies, interaction-based optimization, or additional runtime mechanisms to set phase-dependent speed.AdaTempo instead treats shared relative tempo as a self-supervised cue.Source: §I · pp. 1–2
- Demonstrations can differ in absolute speed while sharing relative-tempo patterns across corresponding phases.The paper relates this consistency to phase-specific characteristics: cautious grasping or insertion may tolerate less compression than fast free-space transport.Source: §I · p. 1
- AdaTempo aligns demonstrations, aggregates phase-matched relative tempo while downweighting disagreement, and turns the result into a continuous speedup profile for offline resampling.Standard policies such as ACT or Diffusion Policy are trained on the resampled trajectories, embedding the desired tempo without runtime selection or online retiming.Source: §I · pp. 1–2
- Up to 3.57× speedup, with a stronger success–speed trade-off than the original policies and representative acceleration baselines.Evaluations cover simulated and real-world tasks.Source: §I · p. 2
II. RELATED WORK
Prior work addresses imitation-policy timing through learned policies, runtime execution changes, and demonstration-side acceleration. AdaTempo differs by using cross-demonstration relative tempo after phase correspondence to derive its speedup profile.
A. Visuomotor Imitation Learning
- Visuomotor policies can reproduce the execution tempo in demonstrations, while inference-latency methods target computation rather than physical task execution time.AdaTempo temporally resamples demonstrations before policy training.Source: A. Visuomotor Imitation Learning · p. 2
B. Acceleration of Imitation Learning Execution
- Acceleration methods change policy execution at runtime or alter demonstrations and training targets before deployment.Runtime methods adjust speed using motion complexity, tracking error, latency, or reachability; demonstration-side methods include proxy-based downsampling, semantic cues, and optimized speed transformations.Source: B. Acceleration of Imitation Learning… · p. 2
C. Temporal Correspondence and Relative Tempo
- Prior alignment work uses DTW for sequence correspondence or label propagation, while motor-control studies motivate comparing normalized relative timing across speeds.AdaTempo uses DTW to match demonstrations and derive a continuous speedup profile, rather than propagating predefined labels.Source: C. Temporal Correspondence and… · p. 2
III. ADATEMPO
AdaTempo transforms demonstrations into an accelerated dataset while preserving task ordering. It aligns relative-tempo profiles to canonical phase and uses their shared structure to resample demonstrations for standard policy training.
III. ADATEMPO
- The pipeline aligns demonstrations of different durations to canonical phase, then uses relative-tempo consensus and agreement to form a continuous speedup profile for resampling.The resulting trajectories are used for standard visuomotor policy training.Source: §III · p. 2, Figure 2 · p. 3
A. Phase-Aligned Relative Tempo
- Arc length represents the fraction of cumulative demonstrated motion completed, but equal arc-length positions need not correspond to the same task phase.A medoid trajectory is selected as the canonical reference using median distances to the others.Source: A. Phase-Aligned Relative Tempo · p. 3
- AdaTempo uses progress-regularized DTW to align each trajectory to the medoid, producing a monotonic canonical phase from matched reference progress.The progress term discourages matches between distant task regions without forcing equal timing; the matched progress sequence is made nondecreasing and normalized to 0–1.Source: A. Phase-Aligned Relative Tempo · p. 3
- Within each demonstration, phase-bin tempo is ranked so larger values mark relatively faster phases while removing the episode’s global tempo scale.Tempo is estimated inversely from smoothed phase-bin occupancy, then interpolated over canonical phase.Source: A. Phase-Aligned Relative Tempo · p. 3
B. Consensus-Guided Continuous Acceleration
AdaTempo aggregates aligned relative-tempo profiles into a phase-wise consensus and disagreement measure. It maps these to a continuous, bounded speedup profile and smooths its changes before offline resampling.
a) Tempo consensus and confidence
- At each phase bin, the median aligned relative tempo defines consensus, while median absolute deviation measures cross-demonstration disagreement.The disagreement measure is used to form confidence in the repeatability of a relative-tempo tendency, not task success.Source: B. Consensus-Guided Continuous… · pp. 3–4
b) Continuous speedup mapping
- The continuous speedup profile combines phase-wise consensus and confidence within task-specific minimum and maximum speedup bounds.Near trajectory boundaries, where alignment and occupancy estimates are less reliable, speedup is set to cmin.Source: B. Consensus-Guided Continuous… · p. 4
- A forward-window lookahead anticipates cautious phases, and asymmetric slew limits smooth frame-to-frame speedup changes before offline resampling.The profile can reduce speedup more rapidly than it increases it.Source: B. Consensus-Guided Continuous… · p. 4
C. Offline Resampling and Policy Training
AdaTempo constructs accelerated trajectories by advancing continuously through each source demonstration, with step sizes varying by phase. It interpolates observations and targets so standard policies can train on the resampled data offline.
C. Offline Resampling and Policy Training
- Virtual source time advances until the original trajectory’s final frame, with each accelerated step mapped to a continuous position on the source timeline.The process starts at s_i,0 = 0.Source: C. Offline Resampling and Policy… · p. 4
- Resampling allocates denser samples to relatively cautious phases and larger temporal steps to phases with greater acceleration potential.Source: C. Offline Resampling and Policy… · p. 4
- At each accelerated step, the observation comes from s_i,k and the target from s_i,k+1, using interpolation for continuous quantities, SLERP for rotations, and the nearest frame for images.The resulting trajectories preserve task order and original control frequency, then train standard policies such as ACT or DP; deployment needs no extra tempo-selection module or online retiming.Source: C. Offline Resampling and Policy… · p. 4
IV. EXPERIMENTS
The experiments test AdaTempo across policy backbones and simulated and real-world settings, compare its success–speed trade-off with acceleration baselines, assess relative-tempo consistency, and examine which components support reliable acceleration.
IV. EXPERIMENTS
- The evaluation asks whether AdaTempo accelerates policies across different backbones and simulated and real-world settings.Source: §IV · p. 4
- It compares AdaTempo’s success–speed trade-off with uniform and adaptive acceleration baselines.Source: §IV · p. 4
- It tests whether demonstrations share within-task relative-tempo structure and whether that structure reproduces across demonstrators.Source: §IV · p. 4
- It investigates which components matter for reliable acceleration.Source: §IV · p. 4
A. Experimental Setup
The setup spans five simulation tasks and six real-world tasks, using ACT or DP according to the task suite. It compares original policies and acceleration baselines under task-specific implementation choices and defined evaluation protocols.
A. Experimental Setup
- The simulation suite pairs ACT with 50 demonstrations for ALOHA Transfer Cube and Insertion, and DP with 200 demonstrations for robomimic Can, Lift, and Square.ACT predicts 14-D absolute joint-position targets; DP predicts absolute Cartesian end-effector poses.Source: A. Experimental Setup · p. 4, Figure 3 · p. 5
- The real-world suite uses separate ACT policies for six tasks on an AgileX ALOHA bimanual platform, trained from about 50 teleoperated demonstrations per task, or 60 for Conveyor Fast.Policies use RGB from two wrist cameras and one overhead camera, plus robot proprioception; trials are timed from first commanded motion to task success.Source: A. Experimental Setup · pp. 4, 6, Figure 4 · p. 5
- Comparisons include the original policy, uniform acceleration, and adaptive methods; real-world tests compare AdaTempo with original ACT and DemoSpeedup, while SAIL is simulation-only.The baselines include ACT-2× or DP-Fast, DemoSpeedup, and SAIL; all learned methods use the same source demonstrations.Source: A. Experimental Setup · pp. 4–5
- Acceleration bounds are task-specific, selected from candidate pairs using held-out validation rollouts, and fixed before final evaluation.Table I reports the bounds used in the experiments.Source: A. Experimental Setup · p. 5, Table I · p. 6
- Simulation results average three independently seeded policies over 100 rollouts per task and method, reporting success rate and speedup-over-demonstration (SOD).SOD is mean demonstration duration divided by mean successful-rollout duration; failed rollouts count toward success rate but not SOD.Source: A. Experimental Setup · p. 5
B. Simulation Results
Across five simulation tasks, AdaTempo provides a stronger success–speed balance than the compared methods, with advantages spanning ACT and DP tasks rather than relying on maximum speed alone.
B. Simulation Results
- 75.5% success rate and 2.83× mean SOD are AdaTempo’s highest average success and mean speedup across the five simulation tasks; DemoSpeedup has 68.1% success and 2.86× mean SOD.Table II reports results across the ACT and DP suites.Source: B. Simulation Results · p. 6
- Across ACT tasks AdaTempo has the highest success on Transfer Cube and Insertion; across DP tasks it preserves perfect Lift success at 3.28× SOD, has Can’s highest SOD, and Square’s highest success.Some competing methods are faster on individual tasks but often with larger success degradation.Source: B. Simulation Results · p. 6
C. Real-World Results
AdaTempo maintains strong real-world task success under accelerated execution. Across demonstrations, relative-tempo patterns show consistency across demonstrators and stronger agreement with DTW phase correspondence.
C. Real-World Results
- 77.8% aggregate success across six real-world tasks accompanies a 2.02× mean task-wise completion-time speedup over ACT.AdaTempo’s success rate exceeds ACT (58.3%) and DemoSpeedup (56.7%); its speedup is comparable to DemoSpeedup’s 1.95×.Source: C. Real-World Results · p. 6
- AdaTempo achieves the best or tied-best success and shortest completion time on four of six real-world tasks.Its advantage is particularly pronounced on Conveyor Fast, where it maintains high reliability at the fastest completion time.Source: C. Real-World Results · p. 6
D. Relative-Tempo Consistency Across Demonstrations
- Across 60 Conveyor Fast demonstrations, the three demonstrators’ consensus profiles closely follow the pooled consensus, with pairwise Spearman correlations of 0.983, 0.941, and 0.955.This consistency indicates that the relative-tempo structure is not specific to one demonstrator’s execution style.Source: D. Relative-Tempo Consistency Across… · p. 6
- Across all 11 tasks, median correlations range from 0.662–0.885 with arc-length-only correspondence and 0.861–0.952 with DTW, which improves every task.The correlations measure agreement in the ordering of faster and slower phases.Source: D. Relative-Tempo Consistency Across… · p. 6
E. Ablation Studies
Ablations on ALOHA simulation tasks examine how consensus, confidence weighting, temporal regularization, and RBD-style augmentation relate to performance. The reported findings favor the first three components for reliability, while RBD-style augmentation provides no consistent benefit.
Ablation Studies
- The focused ablations compare the full method with four variants on Transfer Cube and Insertion, which differ in precision demands.Table IV covers ALOHA simulation tasks.Source: E. Ablation Studies · p. 6, Table IV · p. 6
- The ablation variants remove confidence weighting, remove temporal regularization, or add offset-resampling variants following the replicate-before-downsample principle.Removing temporal regularization removes the lookahead soft minimum and asymmetric slew limits, directly using the instantaneous speed curve.Source: E. Ablation Studies · pp. 6–7
- Cross-demonstration consensus, confidence weighting, and temporal regularization each support reliability, while removing confidence weighting or temporal regularization yields a more aggressive success–speed trade-off.A single demonstration’s tempo profile reduces success, particularly on Insertion; removing confidence weighting raises SOD but lowers success on both tasks.Source: Table IV · p. 6, E. Ablation Studies · p. 7
- RBD-style augmentation provides no consistent benefit in the experimental setting.Continuous nonuniform resampling samples fractional source times, so offset variants add interpolated state–action targets rather than simply recovering demonstrated states omitted by fixed-stride downsampling.Source: E. Ablation Studies · p. 7
V. CONCLUSION AND LIMITATIONS
Across five simulated and six real-world tasks, AdaTempo accelerates visuomotor policies while improving success. Its relative-tempo cue is empirical, and task-specific bounds and physical limits constrain acceleration.
V. CONCLUSION AND LIMITATIONS
- Across five simulated and six real-world tasks, AdaTempo reached up to 3.57× SOD and the highest average simulation success at mean 2.83× SOD.The reported mean simulation speed for the highest average success was 2.83× SOD.Source: §V · pp. 7–8
- In the real world, aggregate success rose from 58.3% to 77.8% over ACT while tasks were completed 2.02× faster on average.Source: §V · pp. 7–8
- Relative tempo is an empirical cue, not a direct estimate of maximum feasible speed; shared teleoperation habits and diverse strategies can affect its usefulness.A single medoid and consensus may be inadequate for unrelated pauses, ambiguous correspondences, or qualitatively different strategies.Source: §V · p. 8
- Acceleration bounds are task-specific and selected with held-out validation; resampling cannot remove physical limits, and learned tempo cannot adapt online.Aggressive targets may expose tracking error, actuator limits, latency, or unmodeled contact dynamics.Source: §V · p. 8