Source-linked AI summary
ACG: Action Coherence Guidance for Flow-based Vision-Language-Action models
Minho Park, Kinam Kim, Junha Hyung, Hyojin Jang, Hoiyeong Jin, Jooyeol Yun, Hojoon Lee, Jaegul Choo
TL;DR
Imitation-trained flow policies can reproduce noise in human demonstrations, reducing action coherence and risking instability or trajectory drift during deployment. ACG uses training-free test-time guidance to steer generation toward coherent actions, with reported gains across simulated benchmarks and real-world tasks.
Problem
Imitation-trained flow policies can retain pauses, jerks, and jitter from human demonstrations, reducing action coherence and contributing to unstable actions or trajectory drift during deployment.
Source: §I · p. 1Method
ACG constructs an intentionally incoherent vector field and guides sampling away from it to generate coherent actions without additional training.
Source: §I · p. 2, A. Guiding Policy with Incoherent… · p. 3Results
ACG outperforms Vanilla GR00T-N1 [2] by 6.7% on RoboCasa, 3.4% on DexMG, 30.8% on Three Strawberries, and 9.6% on average.
Source: B. Benchmark Results · p. 5Takeaways & Limitations
Task-wise results indicate that action coherence is particularly important for fine-grained manipulation, including button pressing and insertion.
Source: B. Benchmark Results · p. 6Generating the guidance vector requires a second forward pass; caching earlier outputs and perturbing only later attention layers reduces overhead to approximately 1.5×, while sufficiency for deeper networks remains open.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Diffusion and flow matching models have emerged as powerful robot policies, enabling Vision-Language-Action (VLA) models to generalize across diverse scenes and instructions. Yet, when trained via imitation learning, their high generative capacity makes them sensitive to noise in human demonstrations: jerks, pauses, and jitter which reduce action coherence. Reduced action coherence causes instability and trajectory drift during deployment, failures that are catastrophic in fine-grained manipulation where precision is crucial. In this paper, we present Action Coherence Guidance (ACG) for VLA models, a training-free test-time guidance algorithm that improves action coherence and thereby yields performance gains. Evaluated on RoboCasa, DexMimicGen, and real-world SO-101 tasks, ACG consistently improves action coherence and boosts success rates across diverse manipulation tasks. Code and project page are available at https://github.com/DAVIAN-Robotics/ACG and https://DAVIAN-Robotics.github.io/ACG , respectively.
I. INTRODUCTION
Imitation-trained flow policies can reproduce pauses, jerks, and jitter in demonstrations, reducing action coherence and risking unstable manipulation or trajectory drift. ACG applies training-free test-time guidance and improves results across simulated and real-world tasks.
I. INTRODUCTION
- Imitation learning can preserve demonstration noise such as pauses, jerks, and jitter, degrading action coherence, defined as smoothness and consistency across successive actions.During deployment, unstable actions can disrupt critical moments, while small action noise can accumulate into trajectory drift.Source: §I · p. 1
- ACG disrupts temporal communication by replacing self-attention maps with identity maps, then guides sampling opposite the resulting incoherent denoising vector.This training-free test-time method is intended to produce more coherent action sequences.Source: §I · pp. 1–2, Figure 1 · p. 1
- ACG improves over Vanilla GR00T-N1 [2] on RoboCasa and DexMimicGen benchmarks and real-world SO-101 pick-and-place, with gains of 23.1% on button pressing, 11.8% on insertion, and 28.8% on real-world pick-and-place.Source: Figure 2 · p. 1, §I · p. 2
- ACG reports gains of 23.1% on button pressing, 11.8% on insertion, and 28.8% on real-world SO-101 pick-and-place.These are highlighted as fine manipulation tasks in evaluation across RoboCasa, DexMimicGen, and SO-101.Source: Figure 2 · p. 1, §I · p. 2
A. VLA Models with Flow Matching Policy
The paper situates ACG among flow-based VLA policies and prior guidance methods, then motivates improving coherence within action chunks. It outlines the flow-matching policy and the sources of temporal coherence and inference trade-offs.
A. VLA Models with Flow Matching Policy
- Flow-based VLA models pair a vision-language backbone with an action head and use visual observations, language instructions, and robot state to predict actions.The paper focuses on generative flow-based VLA models such as GR00T-N1, π0, and SmolVLA.Source: A. VLA Models with Flow Matching Policy · p. 2
B. Guidance for Flow Matching Policy
- Classifier-Free Guidance and related methods steer generation away from an unconditional or perturbed vector field, but language-condition replacement can destabilize VLA policies.Perturbation guidance uses an intentionally degraded model variant, including variants with dropped units or perturbed attention maps.Source: B. Guidance for Flow Matching Policy · p. 2
C. Action Coherent Policy
- Action chunking generates multiple actions together and can promote smoother trajectories, but does not remove shaky motions within a chunk.Temporal ensembling can be costly for non-autoregressive flow policies; prior guidance methods focus on consistency with preceding chunks, leaving within-chunk coherence underexplored.Source: C. Action Coherent Policy · p. 2
A. Flow Matching Policy
- The flow-matching policy models action chunks conditioned on observations and language, with each demonstration represented as time-indexed instruction, observation, and action tuples.At inference, it integrates a learned vector field from random noise using forward Euler steps; the integration step is 1/16 in the reported experiments.Source: A. Flow Matching Policy · pp. 2–3
- ACG replaces self-attention maps with identity maps to create an incoherent action sequence, then guides the denoising vector in the opposite direction.The disrupted attention breaks temporal communication among tokens.Source: Figure 3 · p. 3
B. Classifier-Free Guidance for Flow Matching Policy
Classifier-Free Guidance (CFG) is applied at inference time to steer a flow-matching policy’s generation. In the described goal-conditioned setup, it amplifies the language-conditioned direction relative to the unconditional direction.
B. Classifier-Free Guidance for Flow Matching Policy
- CFG combines conditional and unconditional vector fields to steer generation away from the unconditional field and toward the conditional field.The paper’s setup treats language instruction as the condition and leaves observation unconditioned.Source: B. Classifier-Free Guidance for Flow… · p. 3
A. Guiding Policy with Incoherent Action Generation
ACG is a training-free guidance strategy that steers flow policies away from intentionally constructed incoherent behavior and toward coherent action generation. The section also situates evaluation across simulation and real-world manipulation benchmarks.
A. Guiding Policy with Incoherent Action Generation
- ACG contrasts the original policy with an intentionally constructed incoherent variant, steering generation away from unstable, temporally incoherent behaviors toward coherent actions.The method is presented as training-free guidance for precise robotic manipulation.Source: A. Guiding Policy with Incoherent… · p. 3
- The experiments span RoboCasa kitchen manipulation, DexMG bimanual dexterity, and SO-101 real-world pick-and-place tasks.RoboCasa has 24 atomic tasks across seven skill domains; SO-101 examples include strawberries and tic-tac-toe.Source: Figure 4 · p. 4
- The incoherent denoising vector induces a distribution used to formulate the vector field for coherent action generation.The action chunk is generated through forward Euler integration, as in classifier-free guidance.Source: A. Guiding Policy with Incoherent… · p. 4
B. Constructing Incoherent Action Generation Vector
The incoherent reference is constructed by disrupting temporal communication in the flow-matching policy’s self-attention. Replacing its attention map with an identity map disconnects action tokens across timesteps and reduces temporal coherence.
B. Constructing Incoherent Action Generation Vector
- In the transformer-based flow policy, each token represents an action at a timestep, and self-attention lets tokens communicate to maintain coherence.The attention map controls how strongly each action attends to other actions.Source: B. Constructing Incoherent Action… · p. 4
- The incoherent vector field replaces the attention operation with an identity attention map, forcing each action token to attend only to itself.The identity matrix enforces temporal disconnection across value tokens.Source: B. Constructing Incoherent Action… · p. 4
- This disrupted attention produces an action chunk with reduced temporal coherence, providing a reference for guidance toward coherent generation.The paper illustrates the reference in Fig. 3.Source: B. Constructing Incoherent Action… · p. 4
V. EXPERIMENTS
The experiments evaluate whether ACG improves manipulation performance and action coherence in VLA models. They also examine which self-attention layers to perturb and how well ACG transfers across flow-based VLA models.
V. EXPERIMENTS
- The experiments are organized around whether ACG improves VLA manipulation performance, generates coherent actions, and which attention layers and flow-based models work best.The questions are addressed in Sections V-B through V-D.Source: §V · p. 4
A. Experimental Setup
The evaluation uses two simulation benchmarks and real-world SO-101 pick-and-place tasks, with GR00T-N1 as the primary model and several inference-time baselines. The setup also examines performance across RoboCasa skill domains.
A. Experimental Setup
- RoboCasa covers 24 tasks in seven skill domains, DexMimicGen spans bimanual tasks and embodiments, and SO-101 evaluation uses two pick-and-place tasks.Experiments report standard deviations over three runs, with 24 simulation trials and 10 real-world trials; strawberry and tic-tac-toe scores use task-specific increments.Source: A. Experimental Setup · p. 4
- GR00T-N1 is the primary model, trained with 100 demonstrations per simulation task and 50 and 40 demonstrations for the two SO-101 tasks.ACG perturbs the 4th–6th self-attention layers of eight, with guidance scale 3.0.Source: A. Experimental Setup · p. 5
- Baselines include vanilla GR00T-N1, trajectory ensembles, Gaussian smoothing, classifier-free guidance, and white-noise guidance.Smoothing is applied either to final action predictions or intermediate action features.Source: A. Experimental Setup · p. 5
- Across RoboCasa’s seven skill domains, ACG shows strong improvements on fine-grained insertion and button-pressing tasks.Source: Figure 5 · p. 5
B. Benchmark Results
ACG improves benchmark performance over Vanilla GR00T-N1 across simulated and real-world tasks, with particularly strong gains on fine-grained manipulation. Compared with alternative baselines, guidance methods outperform smoothing, while ACG improves over WNG without the same accuracy trade-off.
B. Benchmark Results
- ACG gains 6.7% on RoboCasa, 3.4% on DexMG, 30.8% on Three Strawberries, and 9.6% on average over Vanilla GR00T-N1.The reported gains span simulation and real-world benchmarks.Source: B. Benchmark Results · p. 5
- Action smoothing yields modest gains, but modifying model features or outputs can blur fine-grained action details and limit improvement.Source: B. Benchmark Results · p. 5
- Guidance-based methods achieve greater improvements than smoothing methods, while CFG performs worse than smoothing-based approaches on manipulation tasks.The paper attributes CFG's weaker results to its emphasis on text conditioning rather than coherent action generation.Source: B. Benchmark Results · p. 5
- ACG improves over WNG by 4.3% on RoboCasa and 8.8% on Three Strawberries while disrupting temporal coherence without degrading task-relevant knowledge.WNG's noise injection trades off temporal coherence and accuracy.Source: B. Benchmark Results · pp. 5–6
- Button pressing (+23.1%) and insertion (+11.8%) are fine-grained skills where ACG is especially effective.The authors identify action coherence as particularly crucial for fine-grained manipulation.Source: B. Benchmark Results · p. 6
C. Action Coherence Analysis
The analysis evaluates action coherence using temporal variation and jerk metrics, with measurements restricted to the approach phase toward one strawberry. ACG produces coherent, accurate actions, while other approaches reveal trade-offs between coherence, accuracy, and temporal coordination within action chunks.
C. Action Coherence Analysis
- ATV quantifies temporal coherence, while JerkRMS measures movement smoothness through the root mean square of motor-angle jerk.Jerk is defined as the third derivative of motor angle.Source: C. Action Coherence Analysis · p. 6
- Metrics average across SO-101's six motors and cover only the first 64 approach-phase timesteps toward one strawberry.This restriction avoids later trajectories diverging according to task success or failure.Source: C. Action Coherence Analysis · p. 6
- Vanilla GR00T-N1 exhibits severe jerking and jittering, while ensembles improve coherence but can hesitate between strawberries as averaging blurs distinct action paths.Action and feature smoothing reduce variation but can produce inaccurate actions because they lack a task-specific prior.Source: C. Action Coherence Analysis · p. 6
- CFG does not improve coherence over Vanilla GR00T-N1, and WNG can produce coherent actions while sometimes sacrificing accuracy.The authors argue action coherence is more crucial than goal conditioning for manipulation performance.Source: C. Action Coherence Analysis · p. 6
- ACG targets incoherence within action chunks, whereas Self-GAD targets coherence between chunks; the comparison suggests intra-chunk coherence matters more, and combining methods may improve results further.Appropriate guidance scales and the number and positions of incoherent attention layers also substantially affect performance.Source: Figure 7 · p. 7
- ACG produces the most coherent actions by JerkRMS while maintaining ATV comparable to ensemble methods, and its actions remain accurate and temporally consistent.The authors associate these qualities with higher success rates.Source: C. Action Coherence Analysis · p. 7
D. Ablation Study
ACG outperforms Self-GAD in the reported comparison, and combining them yields the best results. Its performance is robust to several hyperparameter choices and extends across flow-based VLA models, with computational cost and cross-chunk coherence remaining open concerns.
D. Ablation Study
- Guidance scales that are too large degrade performance, 2–6 incoherent layers consistently improve it, and middle or later layers generally work better than early ones.The default guidance scale is 3.0 and the default layer range is 4–6.Source: D. Ablation Study · p. 7
- Both ACG and Self-GAD outperform the baseline, but ACG achieves higher success rates; combining them achieves the best results.The comparison uses Self-GAD’s receding-horizon setup, which predicts 16 actions and executes the first 8.Source: D. Ablation Study · p. 7
- ACG outperforms the vanilla model across tested VLA variations, including π0 and SmolVLA.The method is applicable to models incorporating self-attention layers over action tokens.Source: D. Ablation Study · p. 7
VI. CONCLUSION
- ACG requires a second forward pass; caching earlier outputs and perturbing later layers reduces overhead to approximately 1.5×, while deeper-network sufficiency and cross-chunk guidance remain open.The conclusion identifies extending coherence guidance across chunk boundaries as future work.Source: §VI · p. 7