PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning High-Frequency Continuous Action Chunks in Latent Space

Kunyun Wang, Yuhang Zheng, Yupeng Zheng, Jieru Zhao, Wenchao Ding

arXiv:2605.24931v1cs.RO

TL;DR

At high action frequencies, direct action-space policies can produce imprecise, jittery trajectories and asynchronous inference can disrupt chunk continuity. The paper learns high-frequency action chunks in VAE latent space and introduces Reuse-then-Refine, enabling smoother, more stable execution with fewer stalls in real-world tasks.

  • Problem

    At high frequencies such as 60 Hz, action-space policies and asynchronous chunk switching can produce imprecise, jittery, discontinuous trajectories that hinder smooth robotic control.

  • Method

    The approach learns high-frequency action chunks in a continuous VAE latent space and uses Reuse-then-Refine to improve continuity during asynchronous execution.

  • Results

    Real-world experiments show that latent policies improve smoothness and precision, while RTR enables stable continuous execution with fewer stalls under asynchronous inference.

  • Takeaways & Limitations

    The results support co-designing policy representation and execution for smoother, less-stall high-frequency robotic control in real-world settings.

  • Takeaways & Limitations

    The experiments are conducted at 60 Hz because available sensors, particularly visual sensors, constrain the sampling rate; higher frequencies remain future work.

Abstract

from arXiv · show

Modern robotic policies increasingly rely on action chunking to execute complex tasks in the physical world. While action chunking improves temporal consistency at moderate action frequencies, it becomes insufficient when the action frequency is further increased (e.g., to 60~Hz). At such high frequencies, policies often fail to generate actions that are both temporally smooth and spatially consistent. We address this challenge by shifting high-frequency action learning from the action space to a latent space with variational autoencoder (VAE). This formulation significantly improves both temporal and spatial consistency of high-frequency control. To enable smooth real-time execution, we further introduce Reuse-then-Refine, a chunk-level refine strategy that improves continuity between adjacent action chunks under asynchronous inference. As a result, robots controlled by our policy can execute complex contact-rich tasks continuously, with less pauses and jerky motions. Experiments on three real-world contact-rich robotic tasks show that our approach consistently completes tasks with smooth motions. Our code and data are available at https://github.com/tars-robotics/RTR.

1. Introduction

High-frequency control preserves fine-grained motion but challenges action-space learning with jittery trajectories and discontinuities across asynchronously generated chunks. The paper combines VAE-based latent-space learning with Reuse-then-Refine (RTR) to improve precision, smoothness, continuity, and real-time execution.

  • Motivation: At high frequencies such as 60 Hz, action-space policies often produce imprecise, highly jittery trajectories despite action chunking’s temporal consistency at lower frequencies.The effectiveness of action chunking depends critically on the training and execution frequency.
  • Motivation: High-frequency actions preserve fine-grained motion details and velocity information, enabling continuous trajectories without the repeated acceleration and deceleration of low-frequency control.This avoids the stop-and-go behavior typical of low-frequency execution.
  • Latent-Space Learning: A VAE compresses high-frequency discrete action chunks into low-frequency continuous latent representations that are more amenable to policy learning.The approach targets the high temporal information density and fine-grained spatial variation that burden policy function approximation.
  • Reuse-then-Refine: Reuse-then-Refine is a training-free latent-policy method that combines overlapping executed and newly predicted actions, then refines the sequence through the VAE.It is designed to improve continuity between consecutive action chunks during asynchronous inference.
  • Results: RTR substantially improves chunk-level continuity and reduces execution stalls under asynchronous inference, while the combined approach enables smoother, more stable control with fewer stalls than DP, OFT, and PI0.5.The introduction reports these findings from real-world experiments.

2. Related Work

Prior robotic policies largely use VLA models and asynchronous inference at moderate action frequencies, while high-frequency action chunking for smooth continuous execution remains largely unexplored. Recent work on latent action representations suggests an alternative direction for robotic control and generalization.

  • VLA Models: VLA models transfer knowledge from large, task-agnostic vision–language datasets to robotic manipulation but primarily operate at moderate action frequencies.High-frequency extension for smooth continuous execution remains largely unexplored.
  • Asynchronous Inference: Asynchronous inference strategies overlap policy inference with execution, using approaches such as tactile-feedback tokenization or switching to newly generated action chunks.RDP uses a fast asymmetric tokenizer for closed-loop tactile feedback, whereas SmolVLA switches chunks when inference completes.
  • Latent Action Representations: Latent representations reduce computational cost and improve generation quality in visual content generation, motivating their use for action learning and robotic control.Recent action-learning studies use latent representations for VLA training from internet-collected videos and report strong generalization capabilities.
  • Latent Action Representations: Several robotic-control studies leverage latent action representations to train VLAs from large-scale, internet-collected videos and demonstrate strong generalization capabilities.VQ-VLA introduces a vector-quantized action tokenizer for action learning.

3. Motivation and Analysis

High-frequency actions provide finer spatial resolution and smoother continuous control, but they are harder for learned policies to model accurately. The analysis shows that directly learning high-frequency action chunks can degrade performance, motivating alternative action representations.

  • Action-frequency motivation: Low-frequency actions create distant target poses and repeated acceleration–deceleration, whereas high-frequency actions provide dense targets for smoother continuous control.Lower frequencies implicitly enforce a zero-velocity boundary at every step, causing point-to-point execution and discontinuous motion.
  • Action-frequency motivation: Policies must be trained and executed at high action frequency because temporal mismatch can amplify execution velocity, violate actuator limits, and compromise safety.A lower-frequency generated chunk executed at the demonstration frequency induces a spatial mismatch.
  • Empirical analysis: At 60 Hz, direct high-frequency learning generally degrades performance; OFT and PI0.5 exhibit substantially higher jerk, while DP maintains relatively low Cartesian deviation and jerk.The comparison evaluates DP, OFT, and PI0.5 using demonstrations collected at 60 Hz and downsampled to 15 Hz.
  • Empirical analysis: OFT is especially affected because discrete action-tokenization quantization errors become significant when high-frequency action strides are small, increasing deviation and reducing smoothness.The reported effect is pronounced for OFT at 60 Hz.
  • Implication: These findings identify direct action-space learning of high-frequency chunks as substantially more difficult and motivate alternative representations that balance expressiveness with learning stability.The conclusion applies even to state-of-the-art imitation learning methods.

4. Methodology

The method learns high-frequency action chunks in a continuous VAE latent space, then uses Reuse-then-Refine to preserve continuity between asynchronously inferred chunks. Together, these components support precise, smooth, continuous real-time robot execution.

  • Latent-space action learning: The approach shifts high-frequency action-chunk learning from action space to a continuous latent space learned with a variational autoencoder.The latent representation preserves fine-grained motion structure while reducing the complexity of high-frequency action modeling.
  • Latent-space action learning: After VAE training, a fixed-VAE latent policy predicts latent action chunks from observations, which are decoded into high-frequency actions during inference.The latent policy models compact sequences of short-horizon motion patterns rather than independent high-frequency commands.
  • Asynchronous execution: Asynchronous inference can create large discontinuities at chunk boundaries, causing execution gaps, visible stalls, or rollback under high-frequency control.Small spatial strides amplify the effect of temporal misalignment between successive chunks.
  • Reuse-then-Refine: Reuse-then-Refine reuses recently executed actions, concatenates them with non-outdated actions from the new chunk, and refines the result through the VAE.This two-stage process smooths inconsistencies while preserving alignment with the most recently executed actions.
  • Reuse-then-Refine: The VAE refinement adds approximately 2 ms overhead, while the latent policy ensures within-chunk smoothness and RTR ensures continuity across adjacent chunks.Together, they enable smooth and continuous real-time robot execution.

5. Experiment

Experiments evaluate latent-space policies and Reuse-then-Refine (RTR) on three real-world contact-rich tasks and under asynchronous execution. Results show improved smoothness, continuity, precision, safety, and execution speed, while continuous VAE compression outperforms VQ-VAE under moderate compression.

  • Experimental setup: Experiments evaluate Diffusion Policy, OpenVLA-OFT, and PI0.5 on Peel Cucumber, Wipe Vase, and continuous whiteboard writing, using VAE temporal downsampling ratio f = 4.The tasks emphasize trajectory smoothness and continuity in real-world contact-rich manipulation.
  • Metrics: Dataset-based evaluation measures action precision with xyz MAE in millimeters and rpy MAE in degrees, while acceleration and jerk quantify motion smoothness.Real-robot evaluation additionally reports exceed count for actions above the 120 mm/s safety threshold, equivalent to a 2 mm per-step limit at 60 Hz.
  • Latent-space policies: Latent-space training consistently improves action smoothness and generally reduces deviation versus high-frequency action-space training, except for PI0.5 Cartesian position error.The comparison uses DP, OFT, and PI0.5 on the real-world whiteboard writing task.
  • Latent-space policies: Across three real-robot tasks, latent policies consistently achieve lower jerk and exceed count than action-space counterparts, producing smoother, more stable execution with fewer stalls.For OFT, the latent policy also yields cleaner written lines and faster end-to-end execution.
  • Asynchronous execution: RTR consistently improves chunk-level continuity and reduces boundary gaps, achieving the lowest jerk and exceed count among compared asynchronous-inference methods.Unlike RT-C, which can degrade continuity when directly applied in latent space, RTR mitigates rollback and execution stalls at chunk boundaries.
  • Latent compression and representation: Continuous VAE achieves lower deviation than VQ-VAE, while moderate compression improves precision and largely preserves smoothness; overly aggressive compression degrades both.These trends are consistent with the behavior observed for latent policies in additional figures.

6. Conclusion · Appendix

The paper presents a latent-space method for learning high-frequency action policies that yields more precise and smoother action chunks than direct action-space learning. It also introduces Reuse-then-Refine to improve chunk continuity during stable real-time execution with asynchronous inference.

  • 6. Conclusion: The method learns high-frequency action policies in latent space.
  • 6. Conclusion: Latent-space learning produces action chunks that are more precise than those learned directly in action space.
  • 6. Conclusion: Latent-space learning produces smoother action chunks than direct action-space learning.
  • 6. Conclusion: Reuse-then-Refine improves chunk-level continuity during asynchronous inference.
  • 6. Conclusion: The RTR strategy is designed to support stable real-time execution.
  • 6. Conclusion: Extensive real-world experiments demonstrate improved action smoothness and precision for latent policies.

A. Limitations and Future Works · B. Experimental Details

The work achieves smooth, continuous real-time execution for high-frequency action chunks but identifies limitations involving sensor-constrained frequency and RTR’s untested use with action-space policies. These limitations motivate exploring higher frequencies and broader refinement applicability.

  • A. Limitations and Future Works: The approach enables smooth and continuous real-time execution on physical robots, while several limitations remain.The work proposes Reuse-then-Refine to improve chunk-level continuity under asynchronous inference.
  • A. Limitations and Future Works: Experiments are conducted at 60 Hz because available sensors, particularly visual sensors, constrain sampling rates.This frequency provides sufficient temporal resolution to encode velocity information and support smooth continuous execution.
  • A. Limitations and Future Works: Higher frequencies such as 90 Hz or 120 Hz may introduce new challenges and opportunities.They would provide denser temporal information and finer spat
  • A. Limitations and Future Works: Further increasing action frequency is a promising direction beyond the sensor-constrained 60 Hz setting.The passage frames higher-frequency operation as an area for future research rather than a demonstrated result.
  • A. Limitations and Future Works: RTR improves chunk-level continuity for latent-space policies, but its applicability to policies trained directly in action space has not been explored.For latent policies, RTR leverages the pretrained VAE used during policy training.
  • A. Limitations and Future Works: Applying RTR to action-space policies would require training an additional VAE solely for refinement during execution.The passage states that training a VAE is significantly less expensive than t

B.1. Experiment setup details … B.4. Experimental hyperparameters

The experiments evaluate latent-space high-frequency action learning at 60 Hz across three contact-rich manipulation tasks and three imitation-learning policies. The setup uses matched training configurations, a frozen pretrained VAE, and real-time deployment on a networked robotic system.

  • B.1. Experiment setup details: The method uses a VAE with temporal downsampling ratio f = 4, followed by a latent policy predicting low-frequency continuous latent actions.Both original and latent policies use the same number of optimization steps unless otherwise specified.
  • B.1. Experiment setup details: 60 Hz is treated as the high-frequency action setting, with prediction horizon H = 48 for all models.This horizon covers approximately one second of demonstrated motion.
  • B.1. Experiment setup details: Experiments use Peel Cucumber, Wipe Vase, and Write Board, three real-world contact-rich tasks emphasizing trajectory smoothness and continuity.The tasks involve cucumber peeling, vase stain removal, and continuous whiteboard line drawing.
  • B.2. Hardware: Data collection and evaluation use a 7-DOF xArm 7 with a Robotiq 2F-85 adaptive gripper.Collection uses an NVIDIA RTX 3060 workstation, while remote inference uses an NVIDIA RTX 4090 server over network communication.
  • B.3. Base models: The evaluated base policies are Diffusion Policy, OpenVLA-OFT, and PI0.5, all of which use action-chunk representations.Diffusion Policy is described as an early action-chunking method, while OpenVLA-OFT and PI0.5 are vision–language–action policies.
  • B.3. Base models: For each base model, the study compares an action-space variant with a latent-space variant.The supplied passage introduces the comparison between the original policy trained directly on high-frequency actions and the latent-space alternative.
  • B.4. Experimental hyperparameters: The VAE is trained independently on each policy’s dataset, then frozen and reused for subsequent latent-space policy training.Its training hyperparameters are summarized in Table 7.
  • B.4. Experimental hyperparameters: Action-space and latent-space variants share architecture, optimizer, learning-rate schedule, batch size, training duration, and preprocessing; only the action representation differs.The configurations are identical across the compared variants for Diffusion Policy, OpenVLA-OFT, and PI0.5.

B.5. Asynchronous real-robot experiment details · C. Supplementary Experiments

The asynchronous real-robot setup uses a 48-step high-frequency prediction horizon and a 24-action latency window. This 0.4 s window accommodates end-to-end inference latency, whose network component is dominated by transmitting high-dimensional visual observations.

  • B.5. Asynchronous real-robot experiment details: The high-frequency policy uses a prediction horizon of 48, spanning 0.8 s in the demonstration data.This matches the temporal coverage of a horizon of 12 under 15 Hz policies.
  • B.5. Asynchronous real-robot experiment details: A horizon of 48 at high frequency matches the temporal coverage of a horizon of 12 under 15 Hz policies.The 15 Hz horizon is described as a commonly used low-frequency baseline.
  • B.5. Asynchronous real-robot experiment details: The asynchronous execution uses a latency window size of 24 actions.After 24 actions from the current chunk, including previously dropped outdated actions, a new policy inference is triggered.
  • B.5. Asynchronous real-robot experiment details: A window size of 24 covers a temporal span of 0.4 s during real-robot execution.This duration is reported as sufficient to accommodate end-to-end inference latency.
  • B.5. Asynchronous real-robot experiment details: Network latency includes transmitting observations to a remote policy server and sending the predicted action chunk back to the robot.Observation transmission dominates because the observations include high-dimensional visual inputs such as images.

C.1. Impact of latent compression on policy precision and smoothness … C.4. Generalizability of the Latent Representation

Moderate latent compression improves precision while preserving smoothness, but excessive compression degrades both, with the tolerable ratio depending on the policy. Across interpolation comparisons, additional contact-rich tasks, and LIBERO simulations, latent policies improve execution consistency without compromising task-level generalization.

  • C.1. Impact of latent compression on policy precision and smoothness: For PI0.5, increasing the downsampling ratio from f = 1 to f = 8 reduces Cartesian prediction deviation, but f = 16 increases deviation.Moderate compression simplifies the latent representation, whereas excessive compression discards critical high-frequency motion information.
  • C.1. Impact of latent compression on policy precision and smoothness: OFT’s deviation also decreases then increases with compression, but its turning point is f = 2 rather than PI0.5’s f = 8.OFT’s discrete action tokenization and quantization error make it tolerate only smaller compression ratios.
  • C.1. Impact of latent compression on policy precision and smoothness: For both PI0.5 and OFT, jerk increases monotonically as the downsampling ratio grows, revealing a precision–smoothness trade-off.Higher compression enlarges temporal gaps between reconstructed actions and amplifies high-order temporal derivatives.
  • C.2. Comparison with interpolation: Interpolation with DP produces higher jerk and exceed count than latent representations, causing frequent pauses and slower execution.Training at lower action frequency and interpolating at execution time does not resolve the underlying consistency issue and can reduce precision.
  • C.3. Dataset-based benchmarking on additional tasks: Across Peel Cucumber and Wipe Vase, latent-space policies consistently achieve higher precision than action-space counterparts, while Peel Cucumber also shows lower jerk.Reduced Cartesian jerk is especially relevant because smoothness affects contact stability and execution safety.
  • C.4. Generalizability of the Latent Representation: LIBERO experiments evaluate original and latent versions of ACT and PI0.5 across four task suites using task success rates.The suites are LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-10.
  • C.4. Generalizability of the Latent Representation: Latent versions achieve success rates comparable to, and sometimes higher than, their original counterparts, indicating preserved task-level generalization.ACT is not evaluated on LIBERO-Goal because the implementation does not condition on language instructions, which that suite requires.

C.5. Reconstruction Error of the VAE · C.6. OOD Generalization of Reuse-then-Refine

The VAE reconstructs real-world high-frequency action chunks with submillimeter deviations while preserving spatial structure. RTR handles reused asynchronous chunks with only modest additional prediction error, retaining submillimeter accuracy alongside improved continuity.

  • C.5. Reconstruction Error of the VAE: C.5. Reconstruction Error of the VAE: The VAE introduces only small reconstruction errors across all three real-world tasks.Mean deviations are measured along each Cartesian axis.
  • C.5. Reconstruction Error of the VAE: C.5. Reconstruction Error of the VAE: Peel Cucumber reconstruction errors are ∆x = 0.37 mm, ∆y = 0.11 mm, and ∆z = 0.17 mm.All three deviations are submillimeter.
  • C.5. Reconstruction Error of the VAE: C.5. Reconstruction Error of the VAE: Wipe Vase reconstruction errors are ∆x = 0.38 mm, ∆y = 0.17 mm, and ∆z = 0.24 mm.All three deviations are submillimeter.
  • C.5. Reconstruction Error of the VAE: C.5. Reconstruction Error of the VAE: Write Board reconstruction errors are ∆x = 0.50 mm, ∆y = 0.28 mm, and ∆z = 0.12 mm.These results show that the VAE preserves fine-grained spatial structure while providing a smoother, more compact latent representation.
  • C.6. OOD Generalization of Reuse-then-Refine: C.6. OOD Generalization of Reuse-then-Refine: RTR reuses overlapping executed actions, concatenates them with the non-outdated predicted portion, and refines the sequence through the VAE.This produces inputs that differ from the exact action chunks used during VAE training.
  • C.6. OOD Generalization of Reuse-then-Refine: C.6. OOD Generalization of Reuse-then-Refine: With RTR, open-loop errors are ∆x = 0.62 mm, ∆y = 0.17 mm, and ∆z = 0.58 mm, versus ∆x = 0.48 mm, ∆y = 0.14 mm, and ∆z = 0.42 mm without RTR.RTR slightly increases open-loop prediction error, but the error remains submillimeter and closed-loop results show improved inter-chunk continuity without reduced task success rates.

D. Demo Videos

The demo videos qualitatively compare action representations and execution strategies across contact-rich tasks. They show that latent-space training improves smoothness, while Reuse-then-Refine reduces asynchronous discontinuities, stalls, rollback, jitter, and latency.

  • Effect of Reuse-then-Refine: RTR reduces discontinuities during asynchronous chunk switching, enabling smoother transitions and reducing execution stalls and occasional rollback.Naive asynchronous strategies introduce large gaps during chunk switching.
  • Effect of latent representation: Latent-space training substantially improves trajectory smoothness over direct high-frequency action-space training for OFT.Action-space OFT exhibits significantly higher jitter.
  • Effect of Reuse-then-Refine: For OFT, RTR further reduces jitter and lowers end-to-end latency under asynchronous execution.This observation is consistent with the DP demonstrations.
  • Effect of Reuse-then-Refine: For PI0.5, RTR achieves the lowest end-to-end latency and highest continuity among naive asynchronous execution, RT-C, and RTR.RT-C improves continuity but residual rollback and execution stalls remain; RTR further mitigates them.
Loading 2605.24931v1…