Commit While Futures Agree: Consequence-Aware Adaptive Action Chunking for Robot Manipulation

• Fixed execution horizons can outlast reliable action near contact or under distribution shift, while frequent replanning adds inference cost and reduces motion consistency.Source: §1 · p. 1
• Action-space ambiguity does not directly indicate execution risk: different trajectories may share an outcome, while similar actions can lead to different outcomes.Source: §1 · p. 2
• CA3C compares candidate chunks’ imagined futures under shared world-model sampling noise, uses Bayesian change-point inference to set the horizon, and executes the candidate closest to consensus.Source: §1 · p. 2, Figure 1 · p. 2
• CA3C consistently outperforms corresponding base policies across LIBERO, LIBERO-Plus, RoboCasa, and real-world AgiBot G2 tasks.Source: §1 · p. 2

2.1 Adaptive Action Chunking

• Existing adaptive methods use policy-derived signals, learned horizon selectors, or post-execution prediction–observation checks to make commitment state-dependent.Source: §2.1 · pp. 2–3
• CA3C sets the execution boundary before acting from agreement among candidate-conditioned consequences, rather than action-space confidence or post-execution discrepancies.Source: §2.1 · p. 3

2.2 World Models for Robot Manipulation

• Where prior work uses world models for planning, learning, or evaluation, CA3C uses them at execution time to jointly guide chunk truncation and candidate selection.Source: §2.2 · p. 3

3.1 Preliminaries

• The policy supplies multiple plausible length-H action chunks, and the world model predicts their time-aligned consequences; CA3C leaves both unchanged and executes h≤H actions before replanning.Source: §3.1 · p. 3, §3.2 · p. 3

3.2 Trustworthy Horizon Estimation

• For each replan, CA3C predicts candidate latent futures under shared world-model noise, measures disagreement relative to predicted scene motion, and smooths the curve over three steps.Source: §3.2 · p. 4
• A BIC comparison of a single-line trend with continuous hinge alternatives sets the boundary at faster disagreement growth; without a supported change point, CA3C uses the longest valid prefix.The hinge's nonnegative slope increment prevents a downward bend from counting as increasing risk.Source: §3.2 · p. 5

3.3 Future-Consensus Candidate Selection

• After selecting h⋆, CA3C executes the candidate with minimum consensus cost, equivalent under squared Euclidean distance to the imagined trajectory nearest the candidate mean.This favors a representative consequence rather than an extreme future and introduces no additional threshold.Source: §3.3 · p. 5

4.2 Quantitative Results

• On LIBERO-Plus, average failure reductions are 61.3%, 30.5%, and 45.4% for GR00T, π0, and π0.5; LIBERO OOD reductions are 3.5–17.1%.The LIBERO-Plus gains span all four suites and different baseline strengths.Source: §4.2 · p. 6, Table 2 · p. 6, Table 1 · p. 6
• RoboCasa average failure rates fall 11.6%, 4.6%, and 7.0% for GR00T, π0, and π0.5; LIBERO ID performance is preserved for GR00T and improves for both π policies.RoboCasa gains occur across all four task groups.Source: §4.2 · p. 6, Table 1 · p. 6, Table 3 · p. 7

4.3 Ablation Study

• Adaptive horizon estimation alone reduces failure by 26.9%, while consensus selection alone does not improve the fixed-horizon baseline; the full method reaches 45.4%.The ablation identifies adaptive horizon selection as the core mechanism and consensus selection as a complementary refinement.Source: §4.3 · p. 7, Table 4 · p. 7
• Fixed h=8 and h=10 each reduce average failure by 45.0% relative to h=5, but neither is best across all suites.The suite-dependent optimum means no single global horizon consistently recovers state-dependent commitment's benefit.Source: §4.3 · p. 7, Table 5 · p. 7

4.4 Qualitative Results

• Figure 3 shows h⋆ adapting across an episode: at A it replans after future step 4, while at B it executes the full chunk with h⋆=10.Across 5,141 horizon-only replanning events, 31.7% selected h⋆ between 4 and 9, though h⋆=10 dominated.Source: §4.4 · p. 8, Figure 3 · p. 8, Figure 5 · p. 9
• In matched LIBERO-Goal OOD rollouts, the base policy fails final placement while CA3C completes the task.The contrast concerns the contact-sensitive final stage despite similar early progress.Source: §4.4 · p. 8, Figure 4 · p. 9

4.5 Real-World Experiments

• On AgiBot G2, ACOT+CA3C raises average success from 67.0% to 72.0%, a 15.2% relative reduction in failure rate.The larger Parcel Sorting gain extends adaptive commitment to a fourth policy and a longer, condition-dependent real-world task.Source: Table 6 · p. 8, §4.5 · p. 9, Figure 6 · p. 10

5 Discussion

• CA3C measures how predicted outcomes vary across plausible actions, not confidence in a single prediction or calibrated model error.It supports committing while consequences agree and replanning when they diverge.Source: §5 · p. 10
• CA3C is best suited to settings with diverse plausible chunks and a world model that preserves their control-relevant consequences.Nearly deterministic policies provide little variation to compare; action-insensitive or inaccurate models can cause false agreement or divergence.Source: §5 · p. 10
• Imagined-future prediction adds inference cost, while candidate count, minimum horizon, and temporal smoothing remain structural choices despite BIC.Source: §5 · p. 10

A.1 World-Model Architecture and Training

• The action-conditioned latent video diffusion model predicts 15 future latents per candidate from a clean observation anchor, using action embeddings and temporal attention.The IRASim-XL/2 denoiser operates on latents; RGB decoding is used only for visualization.Source: §A.1 · p. 13
• Three auxiliary training objectives target bidirectional visual prediction, action–state transition consistency, and motion-aware prediction; their heads and masks are removed at inference.Benchmark-specific LIBERO and RoboCasa world models are fine-tuned for 60k steps and remain frozen in CA3C experiments.Source: §A.1 · p. 13

A.2 Complete Inference Procedure

• At each replanning step, the frozen policy samples K = 3 action chunks and the frozen world model predicts their futures from the same observation and initial diffusion noise.The comparison uses the first T ′ = min(T, H, hmax) valid future steps.Source: §A.2 · p. 13
• A width-three temporal mean and BIC compare a linear trend with a continuous hinge model to detect the horizon boundary; CA3C then executes the consensus candidate prefix and replans.If the hinge model is not preferred, it uses the longest valid prefix; hmin = 4 and hmax = 15, while the policy chunk limits realized horizons to h⋆_t ∈ {4, . . . , 10}.Source: §A.2 · p. 13

A.3 Additional Derivations

• The normalized disagreement U_k = D_k/(M_k + ε) lies between 0 and 1 because candidate disagreement is bounded by total predicted motion.Source: §A.3 · p. 13
• Shared task progress can produce substantial predicted motion without increasing normalized disagreement, while separation among candidate futures raises it.Source: §A.3 · p. 13
• Minimizing future-consensus cost is equivalent to selecting the candidate closest to the mean imagined trajectory, without an additional selection threshold.Source: §A.3 · p. 13

B.1 Policy Checkpoints

• Evaluation uses frozen policy parameters and the same checkpoint for each baseline and CA3C variant, with suite-specific GR00T, π0, π0.5, RoboCasa, and AgiBot G2 checkpoints.The π0 and π0.5 RoboCasa models are fine-tuned for 30k steps, and real-world experiments use task-specific ACOT checkpoints.Source: §B.1 · p. 14

B.2 LIBERO ID and OOD

• The 40 standard LIBERO tasks are evaluated under ID and position-shifted OOD conditions, with object-initialization ranges enlarged by 3–9 cm and 100 rollouts per suite per condition.Source: §B.2 · p. 14

B.3 LIBERO-Plus

• LIBERO-Plus covers 24 tasks across four suites, with three Robot Initial States and three Objects Layout tasks per suite and 20 rollouts per task.This yields 120 rollouts per suite.Source: §B.3 · p. 15

Objects Layout

• Objects Layout examples include bowl-placement variants and kitchen tasks such as placing a moka pot on the stove or closing a drawer on a bowl.Source: Objects Layout · pp. 15–16

B.4 RoboCasa Kitchen

• Each of the 24 RoboCasa Kitchen tasks is evaluated with 20 rollouts and grouped as Relocation (10), Container (6), Rotation (5), or Button (3).The grouping follows the interaction-based categorization used by AAC [6].Source: §B.4 · p. 16
• Relocation includes ten tasks, from counter-to-cabinet and sink-to-counter transfers to coffee setup and serving.The listed tasks also include transfers involving the stove, sink, microwave, and cabinet.Source: §B.4 · p. 16
• Container tasks cover opening and closing single and double doors, as well as drawers.Source: §B.4 · p. 16
• Rotation tasks cover turning the sink spout, operating the sink faucet, and turning the stove on or off.Source: §B.4 · p. 16
• Button tasks comprise pressing the coffee button and turning the microwave on or off.Source: §B.4 · p. 17

C Additional Qualitative Results

• Across the matched LIBERO OOD rollouts, π0.5 fails while π0.5+CA3C succeeds from identical initial states.Top-row rollouts show π0.5 failures; bottom-row rollouts show π0.5+CA3C successes, with red and green borders marking the final outcomes.Source: C Additional Qualitative Results · p. 17
• The matched rollout examples include opening the top drawer and putting a bowl inside, and putting cream cheese in a bowl.Source: Figure 7 · p. 17, Figure 8 · p. 17
• Other matched rollout examples put a wine bottle on top of a cabinet or on a rack.Source: Figure 9 · p. 18, Figure 10 · p. 18