Source-linked AI summary
Adaptive Action Chunking at Inference-time for Vision-Language-Action Models
Yuanchang Liang, Xiaobo Wang, Kai Wang, Shuo Wang, Xiaojiang Peng, Haoyu Chen, David Kim Huat Chua, Prahlad Vadakkepat
TL;DR
Fixed inference-time chunk sizes are difficult to select across diverse manipulation tasks because large chunks reduce reactivity while small chunks compromise consistency. AAC uses action entropy from candidate predictions to adapt chunk size dynamically, and experiments report improved performance across simulated and real-world tasks. The method incurs about 20 ms of inference delay with its default sample size of 20.
Problem
Fixed inference-time chunk sizes are difficult to choose across diverse manipulation tasks because large chunks reduce reactivity while small chunks compromise temporal consistency and computational efficiency.
Method
AAC uses action entropy from sampled continuous and discrete action predictions to dynamically select the inference-time chunk size.
Results
AAC improves success rates across simulated and real-world tasks, including a 15% average gain over GR00T on three real-world tasks.
Takeaways & Limitations
AAC provides an inference-time strategy that balances reactivity and consistency across varied robotic manipulation tasks without requiring fixed chunk-size tuning.
Takeaways & Limitations
AAC adds about 20 ms of inference delay with the default sample size of 20, although higher-end GPUs can reduce the computational gap.
Abstract
from arXiv · showhide
In Vision-Language-Action (VLA) models, action chunking (i.e., executing a sequence of actions without intermediate replanning) is a key technique to improve robotic manipulation abilities. However, a large chunk size reduces the model's responsiveness to new information, while a small one increases the likelihood of mode-jumping, jerky behavior resulting from discontinuities between chunks. Therefore, selecting the optimal chunk size is an urgent demand to balance the model's reactivity and consistency. Unfortunately, a dominant trend in current VLA models is an empirical fixed chunk length at inference-time, hindering their superiority and scalability across diverse manipulation tasks. To address this issue, we propose a novel Adaptive Action Chunking (AAC) strategy, which exploits action entropy as the cue to adaptively determine the chunk size based on current predictions. Extensive experiments on a wide range of simulated and real-world robotic manipulation tasks have demonstrated that our approach substantially improves performance over the state-of-the-art alternatives. The videos and source code are publicly available at https://lance-lot.github.io/adaptive-chunking.github.io/.
1. Introduction
Action chunking improves VLA manipulation by reducing compounding errors, but fixed inference lengths are difficult to choose across policies and tasks. AAC addresses this problem by using action entropy to select chunk sizes dynamically.
- Action chunking executes sequences of actions at each observation and can improve success rates by reducing compounding errors.
- Fixed inference chunk sizes vary across GR00T N1.5, π0, and SmolVLA, indicating that one empirically chosen length does not fit all policies or tasks.
- Adaptive chunk-size selection remains challenging because reinforcement-learning approaches require handcrafted value functions or task-specific simulation rewards.
- AAC dynamically selects the inference-time chunk size using action entropy as a cue.
- Experiments span simulated and real-world robotic manipulation benchmarks and report superiority over state-of-the-art alternatives.
2. Related work
Related work treats action chunking as a way to balance consistency and reactivity during VLA inference. Existing approaches use temporal assembly, fixed or asynchronous execution, confidence signals, or candidate-scoring strategies.
- Diffusion-based VLA models use action chunking during training to learn sequences of future actions across multiple timesteps.
- At inference-time, prior methods balance consistency and reactivity through temporal assembly, fixed chunk sizes, or asynchronous execution.
- DeepConf uses model output confidence for Best-of-N selection but is limited to large language models.
- BID selects action chunks from multiple candidates using designed backward-coherence and forward-contrast scores.
3. Preliminaries
The preliminaries describe GR00T N1.5 as a flow-matching VLA with a vision-language conditioning system and diffusion-transformer action head. They also define entropy measures for discrete and continuous action predictions.
- 3.1. GR00T N1.5: GR00T N1.5 combines a vision-language model that extracts features with a diffusion transformer serving as a flow-matching action head.
- 3.1. GR00T N1.5: Given a noised action chunk, robot state, timestep, and sampled noise, the action head learns to approximate the flow vector field ε − A_t.
- 3.1. GR00T N1.5: At inference-time, action chunks are generated through iterative denoising from random noise.
- 3.2. Action Entropy: Discrete action entropy is computed from the policy’s probability distribution over a finite action space.
- 3.2. Action Entropy: Continuous-action uncertainty is represented with Gaussian differential entropy using the covariance determinant.
4. Adaptive Action Chunking
AAC chooses inference-time chunk sizes from action entropy rather than using one fixed length. It estimates entropy from parallel action-chunk candidates, selects a chunk-size change point, and executes the selected prefix to balance reactivity and consistency.
- 4. Adaptive Action Chunking: Large fixed chunks reduce reactivity, while small chunks compromise temporal consistency and computational efficiency across diverse tasks.
- 4. Adaptive Action Chunking: AAC reduces chunk size under higher prediction uncertainty and increases it when the model is more confident.
- 4. Adaptive Action Chunking: AAC estimates discrete-action probabilities and continuous-action covariance from N candidate chunks sampled in parallel.
- 4. Adaptive Action Chunking: Average action entropy combines translation, rotation, and gripper entropies across the h actions in a candidate chunk.
- 4. Adaptive Action Chunking: The selected chunk size is the maximum difference point in average entropy, constrained by a lower bound ξ.
- 4. Adaptive Action Chunking: AAC executes the first h∗ actions and is designed to balance consistency and reactivity throughout each episode.
5. Experiments
AAC is evaluated across simulation benchmarks and real-world manipulation tasks, using action entropy to select an optimal chunk size from candidate action chunks.
- Experiments span simulation benchmarks and real-world manipulation applications to validate AAC’s effectiveness.
- AAC takes N candidate action chunks and evaluates them at the current timestep.
- At each future timestep, AAC computes entropy for continuous translations, rotations, and discrete gripper control.
- For each candidate chunk size, AAC computes the average action entropy across future timesteps.
- AAC selects the optimal chunk size at the maximum-difference point and returns it for the current timestep.
5.1. Simulation Experiments
Simulation experiments evaluate AAC across RoboCasa and LIBERO, comparing it with fixed-horizon and fixed-size GR00T baselines. AAC improves success rates, adapts chunk sizes to task phases, generalizes across backbones, and remains effective under perturbed settings.
- Benchmarks: RoboCasa covers 24 kitchen manipulation tasks, while LIBERO contains four 10-task suites evaluating spatial relationships, object interactions, and task-specific objectives.The LIBERO suites provide 50 human-teleoperated demonstrations per task.
- Experimental setup: The baseline uses a fixed action horizon of 16, whereas AAC selects the inference chunk size h* at each step.Compared fixed-size GR00T models use manually specified inference chunk sizes.
- Quantitative results: 2.3% average success-rate gain across 24 RoboCasa tasks accompanies an increase from 57.6% to 61.4% on Rotation tasks.Button tasks improve more modestly, from 80.3% to 81.3%.
- Quantitative results: 0.9% average success-rate gain on LIBERO raises performance from 94.1% to 95.0%, while LIBERO-Long gains 4%.LIBERO-Long contains multiple subtasks requiring precise operation over longer action trajectories.
- Quantitative results: Fixed chunk sizes are task-dependent: LIBERO-Spatial performs best with h=4, whereas LIBERO-Goal performs best with h=16; AAC consistently outperforms fixed-size baselines on average.AAC uses action entropy to remove manual chunk-size tuning.
- Qualitative results: AAC predicts large chunks during coarse transport and small chunks near grasping, then increases chunk size after grasping to transport the object efficiently.This pattern follows the semantic phases of the task and reflects the intended balance between efficiency and precise control.
- Scalability and robustness: AAC generalizes to the π0.5 backbone on LIBERO and remains effective as position perturbations increase on LIBERO-Pro, although success rates decline for all methods.The method is designed for flow-matching action heads and can be adapted to different backbones.
- Scalability and efficiency: 0.9% success-rate improvement results when entropy samples increase from 1 to 20, after which gains become marginal.The experiments use 20 samples to balance computational efficiency and success rate.
5.2. Real-world Experiments
AAC is evaluated on three real-world manipulation tasks and improves success, precision, generalization, and safety over vanilla GR00T. The experiments include banana placement, emergency-button pressing, and long-horizon toy manipulation.
- Tasks: Three real-world tasks test banana pick-and-place, emergency-button pressing, and long-horizon toy placement followed by drawer closing.The tasks vary in grasp alignment, localization under unseen button positions, and trajectory-wide precision requirements.
- Results: AAC raises average real-world success from 67.0% to 82.0%, a 15% performance gain over GR00T across all three tasks.
- Results: AAC improves banana grasping by better aligning gripper orientation with the target banana.
- Results: AAC generalizes more robustly when emergency-button locations lie near but outside the human-demonstration distribution.
- Results: AAC improves toy-picking success in the long-horizon task through adaptive chunk sizes, while drawer closing has lower precision requirements.
- Accuracy and safety: AAC filters high-entropy actions and stops at an appropriate lowest point, whereas vanilla GR00T can collide with the tabletop.The comparison links dynamically selected chunk sizes with improved action quality, accuracy, and safety.
6. Conclusion
The conclusion presents AAC as an inference-time strategy that uses action entropy to select chunk sizes adaptively. Across simulation and real-world applications, it reports a balance between reactivity and consistency and improved performance over baselines and state-of-the-art alternatives.
- AAC uses action entropy to adaptively select action chunk size at inference-time for robot manipulation.
- AAC uses larger chunks during transportation and smaller chunks during critical manipulation stages, balancing reactivity and consistency.
- Experiments across simulation benchmarks and real-world applications validate AAC’s effectiveness over baseline and state-of-the-art alternatives.
A. Minimum Action Magnitude
The minimum action magnitude constraint selects a dynamic threshold for chunk sizes while supporting temporal consistency and computational efficiency. Action magnitude combines translation, rotation, and gripper motion components.
- The dynamic threshold ξ is defined as the smallest chunk size l whose action magnitude exceeds α.This constraint is introduced to guarantee temporal consistency and computational efficiency; α is set to 3 in the experiments.
- Total action magnitude m(l) sums the translation, rotation, and gripper-motion magnitudes.
- Translational magnitude is the Euclidean norm of the displacement from the action chunk’s start point to the endpoint at chunk size l.
- Rotational magnitude accumulates rotational offsets through sequential quaternion composition.
- Gripper magnitude equals 1 when the gripper state switches and 0 otherwise.
B. Complete Experiment Results in RoboCasa
Complete RoboCasa results are reported for all 24 tasks and across different chunk sizes. AAC achieves the best average overall performance, although task-specific chunk-size optimization remains incomplete.
- Complete success-rate results cover all 24 RoboCasa tasks.
- AAC achieves the best average overall performance across the RoboCasa tasks.
- Results are also reported for different chunk sizes, while optimal task-specific chunk-size selection remains an open improvement area.