PaperlayerView exact revision on arXiv

Source-linked AI summary

Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks

Guoheng Sun, Chen Chen, Jin Wang, Ang Li, Teresa Lv

arXiv:2609.36471v1cs.ROcs.AIcs.CV

TL;DR

Long action chunks amortize inference, but later actions become less reliable when they remain conditioned on stale observations. STAIRCASE POLICY streams denoising and execution while refreshing unexecuted actions from new observations. Across the reported benchmarks, S-WAM improves throughput and task performance, reaching 292.7 executed actions per second at comparable accuracy to vanilla execution.

  • Problem

    Longer executed horizons reduce action reliability because later actions remain conditioned on stale observations as errors accumulate and scenes change.

    Source: §1 · p. 2

  • Method

    STAIRCASE POLICY streams staggered action sub-chunks and re-predicts the future latent from new observations to refine all unexecuted actions.

    Source: §1 · p. 2

  • Results

    Across simulation benchmarks and real-robot tasks, S-WAM improves throughput and task performance; against vanilla execution at Hexec=10, throughput rises from 80.9 to 292.7 executed actions per second (3.62×).

    Source: §4.3 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    Streaming generation and observation-conditioned refinement provide an effective way to execute longer action horizons without repeatedly invoking the full policy.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds further inference overhead to already expensive iterative action generation. Action chunking can amortize this cost over multiple actions, yet performance degrades over long execution horizons because later actions remain conditioned on stale observations. We introduce STAIRCASE POLICY, a streaming inference and training framework that turns a flow-matching VLA into a JEPA-style WAM and partitions a large action chunk into sub-chunks at staggered denoising stages. Near-term actions are executed as soon as they become available, while later actions continue to be refined. At each sub-chunk boundary, the future latent is re-predicted from the latest observation and used to update all unexecuted actions, enabling long-horizon execution without repeated full policy inference. The resulting future-prediction error can further serve as a signal for adaptive chunking. S-WAM achieves 97.7% on LIBERO and 87.9% on LIBERO-Plus, and improves performance across multiple policy backbones and real-robot tasks. It reaches 292.7 executed actions per second, $3.62\times$ the throughput of conventional execution at comparable accuracy, while reducing time-to-first-action from 123.6 to 73.3 ms. With additional inference optimizations, throughput further increases to 642.9 actions per second.

1 INTRODUCTION

WAMs condition action generation on predicted future observations, but prediction adds cost to iterative inference and long chunks leave later actions conditioned on stale observations. STAIRCASE POLICY streams chunk generation and execution while refreshing unexecuted actions from new observations.

1 INTRODUCTION

  • WAMs add a future-prediction module to VLA action generation, while JEPA-style variants predict latent representations rather than future pixels.The paper studies a lightweight predictor whose future representation is jointly conditioned with the current observation for action generation.Source: §1 · pp. 1–2
  • Action chunking amortizes inference across multiple actions, but deployed systems often execute only one to twenty actions before obtaining a new observation and querying again.This limits effective throughput to the executed horizon rather than the full generated chunk.Source: §1 · p. 2
  • Longer executed horizons reduce reliability because tracking and contact errors accumulate, uncertainty grows, and scene changes are absent from stale conditioning.Increasing the executed horizon from 10 to 50 actions reduces success by 12.8 points for LaWAM and 26.8 points for π0.5.Source: §1 · p. 2
  • S-WAM achieves 97.7% on LIBERO and 87.9% on LIBERO-Plus, with 292.7 executed actions per second and 73.3 ms time-to-first-action.With additional optimizations, throughput reaches 642.9 actions per second.Source: §1 · p. 2
  • At sub-chunk boundaries, the policy re-encodes the latest observation, re-predicts the future latent, and advances all unexecuted actions by one denoising step.Near-term sub-chunks execute as they become available while later actions continue to be generated; prediction error can also signal when to terminate a chunk and replan.Source: §1 · p. 2

2 RELATED WORK

Related work addresses future-conditioned action generation, inference acceleration, and the limits of long action chunks. STAIRCASE POLICY distinguishes itself by refreshing the conditioning of unexecuted actions as observations arrive.

2 RELATED WORK

  • WAMs condition actions on predicted future observations in pixel, video, or representation space, while STAIRCASE POLICY repeatedly updates future representations to refine unexecuted actions.Future prediction adds inference overhead, and some recent work questions its necessity at inference time.Source: §2 · p. 3
  • Prior acceleration methods reduce generation cost, model capacity, or inference latency, or stream denoising; these methods condition a chunk on one observation.STAIRCASE POLICY instead reconditions the unexecuted remainder on observations arriving during execution and keeps a shared flow time with different readout depths.Source: §2 · p. 3
  • Prior large-chunk methods terminate early, correct generated chunks, or condition generation on predicted future states; STAIRCASE POLICY updates unexecuted actions as observations arrive.Published LIBERO policies typically execute only 1–20 actions per query, limiting chunking's throughput benefit.Source: §2 · p. 3

3 METHOD

STAIRCASE POLICY turns a flow-matching VLA into a JEPA-style World-Action Model and streams a large action chunk through staggered sub-chunks. It refreshes future-latent conditioning during execution and trains under the same streaming schedule.

3 METHOD

  • STAIRCASE POLICY pipelines large-chunk generation and execution, refining unexecuted actions as new observations arrive rather than repeatedly invoking the full policy.Near-term sub-chunks are executed while later actions continue to be generated.Source: §3 · p. 3, §3.1 · p. 4
  • The design allocates denoising unevenly across the horizon and refreshes visual conditioning while reusing cached vision-language context.Near-term actions are treated as easier to predict; refreshing the lightweight predictor avoids repeatedly invoking the more expensive backbone.Source: §3 · p. 3

3.1 SETUP

  • The host backbone produces semantic context and a latent plan, while a lightweight predictor estimates a future-observation latent that conditions action generation.The flow-matching expert denoises an H-action buffer; the resulting S-WAM conditions on current visual features, predicted future features, and semantic context.Source: §3.1 · pp. 3–4

3.2 THE DENOISING STAIRCASE

  • Each query advances the shared action buffer by one Euler step, then emits the next sub-chunk by extrapolating the velocity estimate while the remainder keeps denoising.Sub-chunk j is emitted after j+1 expert passes; across H executed actions, the schedule uses one backbone pass and K denoising steps.Source: §3 · p. 3, §3.2 · pp. 4–5

3.3 REFRESHING THE FUTURE PREDICTOR

  • At each sub-chunk boundary, the lightweight predictor uses the latest observation to refresh conditioning for every unexecuted buffer position while chunk-start context remains fixed.This lets boundary observations influence subsequent actions without a full policy replan.Source: §3.3 · p. 5
  • The discrepancy between a predicted future latent and its later realization can trigger a new chunk when it exceeds a threshold, making the executed horizon data-dependent.The backbone then plans a new chunk after the current sub-chunk.Source: §3.3 · p. 5

3.4 TRAINING UNDER THE DEPLOYMENT SCHEDULE

  • Training follows the deployment schedule, using self-predicted future latents and supervising action readouts across boundaries so the predictor learns both latent prediction and action-generation utility.Gradients from the action loss propagate through the future latent into the predictor and latent plan.Source: §3.4 · p. 5

4 EXPERIMENTS

Experiments span three simulation benchmarks and nine real-robot tasks. S-WAM combines strong success rates with higher execution throughput, faster training convergence, and improved performance on dynamic tasks.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers LIBERO, LIBERO-Plus, DOMINO, and nine xArm 850 tasks; S-WAM and vanilla share data, initialization, and training configuration within each backbone.LIBERO has four suites, LIBERO-Plus has seven perturbation categories, and DOMINO tests dynamic manipulation; real-robot trials include static and moving-target tasks.Source: §4.1 · pp. 5–6, §4.2 · p. 6

4.2 REAL-ROBOT EXPERIMENTS

  • 44.2 success on dynamic tasks exceeds vanilla at Hexec=50 (25.8) and Hexec=20 (10.8); on six static tasks, S-WAM reaches 72.5.The reported dynamic-task result reflects updating unexecuted actions from new observations; vanilla Hexec=50 accumulates prediction drift over long execution.Source: Figure 3 · p. 6, §4.2 · p. 6

4.3 PERFORMANCE OF S-WAM ACROSS SIMULATION BENCHMARKS

  • 97.7% SR at 292.7 executed actions per second matches the highest SR while delivering 1.8× the throughput of the next-fastest method shown.SRs are taken from original papers; throughput is measured from released checkpoints under a unified protocol.Source: §4.3 · pp. 6–7
  • 87.9% average SR on LIBERO-Plus is 17.3 points above vanilla at Hexec=50; under robot initial-state perturbation, S-WAM scores 76.0 versus 49.7.Table 1 reports robustness across seven perturbation dimensions.Source: Table 1 · p. 7, §4.3 · p. 7
  • At batch size 4, S-WAM reaches 72.5 success after 10k steps versus 32.2 after 25k; total training FLOPs are lower despite 1.87× more FLOPs per step.Across five global batch sizes, S-WAM consistently reaches higher success with fewer training steps, with the largest gains in low-batch regimes.Source: Figure 4 · p. 7, §4.3 · p. 7
  • 3.62× throughput raises execution from 80.9 to 292.7 actions/s while TTFA falls from 123.6 to 73.3 ms at matched accuracy; graph capture and prompt caching reach 642.9 actions/s.The optimization ladder applies graph capture and prompt caching without changing the actions produced.Source: §4.3 · p. 7, Table 2 · p. 8

5 DISCUSSION

S-WAM sustains performance at longer execution horizons, while its chunk length and denoising budget shape the speed–accuracy trade-off. The discussion also compares compact models, adaptive chunking, and transfer across policy backbones.

5.2 CHOICE OF DENOISING STEPS AND SUB-CHUNK SIZE

  • Chunk length is H = K × G; performance stays stable across a broad range and degrades only when the chunk is too long relative to available denoising steps.The experiments use K=5 and G=10, yielding a 50-action chunk, unless otherwise specified.Source: Figure 5 · p. 8, §5.2 · p. 8

5.1 EXECUTION LENGTH AND THE SPEED–ACCURACY TRADE-OFF

  • 97.7 and 96.5 SR at a full 50-action chunk exceed the best vanilla result on either backbone, while vanilla drops as Hexec rises from 10 to 50.With only Hexec varied, the backbones drop from 95.2 to 82.4 and from 93.2 to 66.4, respectively.Source: Figure 6 · p. 8, §5.1 · p. 8

5.3 SMALL MODELS VS. EXECUTION-LEVEL ACCELERATION

  • On LIBERO-Plus, compact models reach 66.9–76.5% versus 87.9% for S-WAM; on DOMINO, they reach 0.9–5.3% versus 19.3% for S-WAM.The paper frames model-size reduction and execution-level acceleration as complementary paths to efficient inference.Source: Figure 7 · p. 8, §5.3 · pp. 8–9

5.4 THE FUTURE-PREDICTION ERROR AS A CHUNKING SIGNAL

  • On LIBERO-Long, p80 reaches 96.2 SR with only a modest throughput reduction, providing an operating point between commit-4 and commit-5.Adaptive chunking replans when future-prediction error exceeds a validation-set percentile threshold, while committing to at least four sub-chunks.Source: Figure 8 · p. 9, §5.4 · p. 9

5.5 TRANSFERABILITY ACROSS POLICY BACKBONES

  • 15.25, 30.15, and 14.05 SR-point gains over vanilla Hexec=50 on LaWAM, π0.5, and FLOWER; versus Hexec=10, throughput rises 2.74×–3.62× and TTFA falls 37.9%–40.7%.π0.5 and FLOWER were not pretrained to predict future latents; the method-side schedule, objectives, and hyper-parameters remained unchanged across backbones.Source: Table 3 · p. 9, §5.5 · p. 9

6 CONCLUSION

STAIRCASE POLICY streams large action chunks and refreshes unexecuted actions from new observations. The paper concludes that this approach improves efficiency while maintaining task performance across simulation and real-robot evaluations.

6 CONCLUSION

  • STAIRCASE POLICY pipelines generation and execution of large action chunks, maintaining sub-chunks at staggered denoising stages and updating unexecuted actions using new observations.The framework avoids repeatedly invoking the full policy while enabling longer execution horizons.Source: §6 · p. 9
  • S-WAM improves action throughput and reduces TTFA without sacrificing task performance across multiple simulation benchmarks and real-robot tasks.The conclusion also reports improved robustness under perturbations and consistent outperformance of compute-matched baselines.Source: §6 · p. 9
  • The paper concludes that streaming generation and observation-conditioned refinement make large action chunks practical for efficient robot control.Source: §6 · p. 9

A TRAINING RECIPES

The appendix specifies shared and host-specific training recipes, then documents how inference speed and optimization effects are measured.

A TRAINING RECIPES

  • Within each host, S-WAM and vanilla share data, initialization, batch size, learning rate, and schedule shape, while differing in mechanism and training steps.Reported checkpoints are selected on the held-out final 1% of training data using the same rule.Source: A TRAINING RECIPES · p. 15
  • LIBERO and LIBERO-PLUS use H=50 with K×G=5×10, while DOMINO uses H=75 with K×G=5×15; loss weights and EMA settings are shared across hosts.The appendix identifies separate tables for LaWAM training-step budgets and transfer-host settings.Source: A TRAINING RECIPES · p. 15, Table 4 · p. 15, Table 6 · p. 16
  • On LIBERO and LIBERO-PLUS, a closed-loop S-WAM training step costs 5.862 TFLOPs per sample versus 3.137 for vanilla, a 1.869 ratio.The dispatch-layer counter omits elementwise operations, normalization, and softmax, making absolute costs low by an unknown margin; the ratio is described as safe.Source: A TRAINING RECIPES · p. 15
  • FLOWER adds a world-model block, compact-model results are reproduced under published LIBERO settings, and ablation and robot runs reuse the LaWAM recipe with adjusted batch sizes.The K and G sweeps vary one factor at a time around K=5, G=10.Source: A TRAINING RECIPES · pp. 15–16

B SPEED-MEASUREMENT PROTOCOL

  • Speed measurements use batch 1 on an exclusively held device, with preprocessing included and model loading and simulator stepping excluded.The protocol discards 20 warmups, times at least 200 iterations, and reports about ±5% run-to-run spread.Source: B SPEED-MEASUREMENT PROTOCOL · p. 16

B.1 OPTIMIZATION-LADDER MEASUREMENTS

  • Optimization-ladder measurements use five repeats per cell, and both optimizations are bit-exact with eager execution.They change production speed but not the actions produced.Source: §B.1 · p. 16

B.2 BACKBONE-TRANSFER MEASUREMENTS

The section measures speed transfer across backbones under matched timing conditions and reports throughput gains for three hosts.

B.2 BACKBONE-TRANSFER MEASUREMENTS

  • For each backbone, S-WAM and both vanilla configurations are timed in the same session on the same exclusively held device, interleaved.Neither variant uses compilation, CUDA graphs, or quantization.Source: §B.2 · pp. 16–17
  • Each backbone runs at its released default, so within-backbone speedup is the primary measure of transfer across backbones.The speed columns add constraints to the preceding protocol to avoid confounding measurement conditions.Source: §B.2 · pp. 16–17
  • Throughput gains over the Hexec=10 baseline are 3.62× on LaWAM, 3.21× on π0.5, and 2.74× on FLOWER.Each cell uses five repeats.Source: §B.2 · p. 17

C FULL LIBERO SPEED AND SUCCESS RATE

The section documents LIBERO success-rate and throughput comparisons, including measurement provenance and a threshold that limits what Figure 1 displays.

C FULL LIBERO SPEED AND SUCCESS RATE

  • LIBERO success rates are taken from source papers, while throughput is measured on released checkpoints at batch 1 using each baseline’s default pipeline and S-WAM in eager mode.Three default pipelines enable compilation or CUDA-graph capture; in eager mode those checkpoints reach 24.9, 22.2, and 27.9 actions per second.Source: C FULL LIBERO SPEED AND SUCCESS RATE · p. 17
  • The reported success rates were not reproduced by the authors.The throughput measurements are their own measurements of released checkpoints.Source: C FULL LIBERO SPEED AND SUCCESS RATE · p. 17
  • Figure 1 starts its y-axis at 94.9%, so five policies below that threshold appear in Table 7 but not in the figure.Table 7 covers LIBERO success rate and measured throughput across methods.Source: C FULL LIBERO SPEED AND SUCCESS RATE · p. 17, Table 7 · p. 17

D PER-SUITE EXECUTION-LENGTH CURVES

The execution-length results are non-monotonic: success can peak at an intermediate horizon, with different patterns across suites and hardware tasks.

D PER-SUITE EXECUTION-LENGTH CURVES

E DETAILED RESULTS FOR THE SMALL-MODEL COMPARISON

This section reports detailed benchmark breakdowns for the small-model comparison and explains how the results are aggregated. Across the more challenging LIBERO-Plus and DOMINO benchmarks, compact-model acceleration falls further behind, while STAIRCASE POLICY improves throughput by extending the execution horizon with the larger host policy.

E DETAILED RESULTS FOR THE SMALL-MODEL COMPARISON

  • The section details the comparison of STAIRCASE POLICY, two vanilla execution-length variants, and three compact-model baselines.The compact models’ training settings are provided in Appendix A.Source: E DETAILED RESULTS FOR THE SMALL-MODEL… · p. 17

E.1 LIBERO

E.2 LIBERO-PLUS

  • LIBERO-Plus entries are averaged across the four suites before the seven-category unweighted mean is computed for the main-paper result.The separate suite breakdowns show that method rankings differ by suite, while averaging their Avg. columns reproduces Table 9’s Avg. exactly.Source: §E.2 · p. 18, Table 9 · p. 19

E.3 DOMINO

  • The DOMINO comparison uses a 75-action chunk, making vanilla full-chunk execution the compute-matched baseline for STAIRCASE POLICY.Thus, DOMINO’s full-chunk rows correspond to Hexec = 75, not 50.Source: §E.3 · p. 20, Table 14 · p. 20

E.4 ACTION THROUGHPUT

  • Compact-model acceleration remains competitive on LIBERO but falls further behind on LIBERO-Plus and DOMINO; STAIRCASE POLICY improves throughput by amortizing the larger host policy over a longer execution horizon.The speed measurements use executed actions per second, computed from p50 latency of a blocking chunk call in eager mode with batch size 1.Source: §E.4 · p. 20

F REAL-ROBOT DATASETS AND DETAILED RESULTS

The real-robot evaluation uses nine manipulation tasks collected on an xArm platform, with static and dynamic variants for three tasks. Results report per-task success rates over 40 trials and unweighted averages for static and dynamic tasks.

Data collection

Observations and actions

  • Each observation combines fixed third-person and wrist-camera images with a 7-dimensional robot state; actions are 7-dimensional pose changes plus a binary gripper command.The state comprises absolute end-effector position, three Euler angles, and gripper opening.Source: F REAL-ROBOT DATASETS AND DETAILED… · p. 20

Tasks

Scene randomization and dynamic tasks

  • Evaluation objects are re-placed using the same procedure as training, while dynamic tasks continuously rotate the target during execution.The turntable period is 16 seconds (22.5°/s); for hang cup, target orientation also changes.Source: F REAL-ROBOT DATASETS AND DETAILED… · p. 21

Detailed results

  • The real-robot evaluation reports per-task success rates over 40 trials, with static and dynamic averages computed as unweighted means over six and three tasks, respectively.S-WAM and vanilla at Hexec=50 are compute-matched; vanilla at Hexec=20 replans 2.5× as often.Source: F REAL-ROBOT DATASETS AND DETAILED… · p. 24, Table 18 · p. 24

Related papers

Loading 2609.36471v1…