Learning Native Continuation for Action Chunking Flow Policies

• Naive action chunking can create discontinuous transitions because of inference delay and flow-policy multimodality.Source: §I · p. 1
• RTC improves continuity through inference-time inpainting but leaves continuation outside the policy, allowing spurious switching, hesitation, and abrupt direction changes.Source: §I · p. 1, B. Trajectory Continuation in Learned… · p. 3
• Legato reshapes the learned flow dynamics so chunk continuation is native to the policy and remains consistent between training and inference under per-step guidance.Source: §I · pp. 1–2
• Its smooth guidance schedule anchors the chunk prefix to known actions, then ramps guidance strength down to zero for fine-grained continuity control.Source: §I · pp. 1–2, Figure 2 · p. 2
• Randomized schedule conditioning supports varying inference delays and lets one policy adapt continuation strength without retraining.Source: §I · p. 2
• Across five manipulation tasks, Legato consistently produces smoother trajectories and shorter task completion times than RTC while suppressing spurious multimodal switching.Source: §I · p. 2

Flow matching

• Flow matching transforms Gaussian noise into fixed-length action chunks through a learned velocity field.At inference, generation integrates the velocity field across denoising time.Source: A. Preliminaries · p. 3, 1) Flow matching · p. 3

Real-Time Chunking

• RTC improves continuity by constraining overlapping regions of newly generated chunks to previously generated actions.Its structured schedule applies full guidance near the boundary before gradually relaxing the constraint.Source: 2) Real-Time Chunking · p. 3

Why per-step guidance matters

• One-shot prefix guidance cannot preserve overlap constraints because the guided actions drift from their reference during multi-step denoising.The drift occurs despite clamping the prefix at initialization.Source: B. Why per-step guidance matters? · p. 4, Figure 4 · p. 4

Legato

• Legato reshapes flow dynamics during training so per-step, schedule-shaped guidance remains consistent with the training objective.The method targets a native continuation property rather than relying solely on inference-time correction.Source: B. Why per-step guidance matters? · pp. 3–4

Native continuation

• Legato requires per-step guidance and training-inference consistency for native action continuation.These requirements motivate the schedule-shaped training path and induced guided dynamics.Source: C. Native Continuation for Action… · p. 4

Action-noise mixture

• The training path mixes reference actions with noise according to a horizon-wise guidance schedule ω.ω provides full guidance near the chunk beginning and ramps toward weaker guidance later.Source: 1) Action-noise mixture · p. 4

Multimodal persistence and smoothness

• Large guidance weights suppress effective transport, making overlap and ramp regions less mutable during denoising.The schedule discourages frequent switching among competing modes and gradually relaxes continuation toward free generation.Source: 1) Action-noise mixture · pp. 4–5

Training-inference consistency

• Legato reshapes the velocity field so the network regresses toward a target consistent with repeated continuation guidance.This preserves the geometric direction of standard flow matching while modifying velocity magnitude.Source: 1) Action-noise mixture · p. 4, 3) Training-inference consistency · p. 5

Inference

• At inference, Legato uses the previous predicted chunk as the continuation reference and integrates the learned dynamics with the training discretization.The reference is aligned into Aref before repeated guidance and denoising updates.Source: 1) Action-noise mixture · p. 4, 3) Training-inference consistency · p. 5

Schedule parameters

• The continuation schedule is uniquely specified by inference delay d and ramp length r.It consists of a full-guidance prefix followed by a ramp section.Source: D. Schedule Randomization and… · p. 5

Schedule randomization

• Randomizing d and r during training exposes the policy to diverse guidance schedules for variable latency and controllable smoothness.The schedule family addresses differences across hardware platforms and inference optimizations.Source: D. Schedule Randomization and… · p. 5

Schedule conditioning

• Schedule conditioning lets inference adapt continuation behavior by changing ω without retraining the model.The action decoder receives the schedule as an additional feature, improving robustness across hardware platforms and inference budgets.Source: D. Schedule Randomization and… · p. 5

Evaluation metrics

• Smoothness is assessed with NLDLJ, NSPARC, and overlap RMSE, for which lower values indicate better performance.NLDLJ captures global jerk-related smoothness, while NSPARC measures frequency-domain velocity-profile smoothness.Source: Table I · p. 6, 2) Evaluation Metrics · p. 6

Tasks and environments

• The evaluation covers five tasks: stacking bowls, pouring, placing items into a box, folding a towel, and opening a drawer.The tasks span rotation- and translation-dominant motions and multimodal grasp or arm selection.Source: 1) Tasks and Environments · pp. 5–6

Evaluation metrics

• The study measures task completion score, completion time, trajectory smoothness, and chunk-overlap RMSE.Completion time captures delays associated with hesitation or spurious action switching, while overlap RMSE evaluates local continuity at chunk connections.Source: 2) Evaluation Metrics · p. 6

Models and training protocol

• RTC and Legato are compared under controlled conditions using the same pretrained checkpoint, datasets, hyperparameters, and training-step count.This protocol is intended to isolate differences between the two methods.Source: 3) Models and Training Protocol · p. 6

Overall results

• Legato consistently outperforms RTC across five physical-robot manipulation tasks.The evaluation reports improvements across task efficiency, smoothness, and success.Source: B. Main Results · p. 6

Task efficiency

• Legato achieves shorter completion times by suppressing multimodal oscillations and reducing hesitation during execution.The effect is especially pronounced when visually similar objects create competing action modes.Source: 1) Task efficiency · p. 7

Trajectory smoothness

• Legato improves smoothness metrics, with statistically significant gains except for NLDLJ.NSPARC and chunk-overlap RMSE improve, while NLDLJ does not consistently reach significance but does not degrade.Source: 2) Trajectory smoothness · p. 7

Task success

• Legato exceeds RTC in task completion scores without sacrificing execution efficiency or trajectory smoothness.The reported task-success improvement supports more reliable real-world manipulation performance.Source: 3) Task success · p. 7

Schedule trade-offs

• Reducing stride strengthens overlap consistency but can degrade global smoothness, while shortening the ramp improves frequency-domain smoothness at weaker overlap alignment.Schedule configurations therefore provide controllable smoothness trade-offs.Source: Figure 7 · p. 7

Training-time RTC comparison

• Legato achieves higher task scores, shorter completion times, and improved smoothness metrics than training-time RTC on the pour task.Both methods are compared under the same experimental settings.Source: C. Comparison with Training-Time RTC · p. 7

Method comparison

• Legato reshapes learned flow dynamics, whereas training-time RTC uses hard prefix conditioning while leaving the underlying flow dynamics unchanged.A zero-length ramp only makes the constraint shapes similar; the learned continuation mechanisms remain fundamentally different.Source: C. Comparison with Training-Time RTC · p. 7

Schedule robustness

• Legato consistently outperforms RTC across schedule configurations and remains superior as stride and delay vary.Execution stride and inference delay act as control knobs for trajectory smoothness.Source: 1) Varying the execution stride s · p. 8, 2) Varying the inference delay d · p. 8

Ablation studies

• Reducing execution stride improves chunk-to-chunk consistency but can degrade whole-trajectory smoothness through accumulated high-frequency components.Shortening the ramp with a small stride reverses this balance by improving NSPARC while weakening overlap consistency.Source: 1) Varying the execution stride s · p. 8

Conditioning

• Explicit schedule conditioning improves adaptation to different continuation regimes induced by varying delay and ramp parameters.Removing the condition row degrades trajectory smoothness and execution stability.Source: 3) Condition Row · p. 8

Kinetix evaluation

• Legato outperforms all compared methods in Kinetix when inference delay is greater than zero.The experiment uses settings identical to Training-time RTC and a UNet-based policy.Source: E. Simulation Results on Kinetix · p. 9

Schedule randomization

• Fixing delay during training degrades performance at other evaluation delays, highlighting the importance of delay randomization.Excluding zero delay from the randomized range can cause near-complete evaluation failure at the first unguided denoising step.Source: E. Simulation Results on Kinetix · p. 9

Deployment

• The deployment recommendation uses the maximum recent delay, half-chunk execution stride, and the remaining horizon as ramp length.For variable denoising-step inference, the no-condition-row variant is recommended.Source: F. Deployment guidelines · p. 9

Training

• Initializing Legato from a strong pretrained base policy yields more stable convergence and reduces excessive-guidance impairment of reactivity.The recommendation is based on observed training behavior rather than a new benchmark comparison.Source: F. Deployment guidelines · p. 9

Limitation

• The current formulation specifies the denoising step at training time, limiting adjustment during inference.More flexible native continuation schemes are identified as future work.Source: §V · p. 9

Evaluation protocol

• Five manipulation tasks use shared initial configurations and, unless noted, randomized object conditions held identical across models.The tasks include pouring, pick-and-place, drawer opening, and towel folding; the supplied protocol also states evaluation counts unless otherwise specified.Source: 2) Pouring (pour) · p. 12, 3) Pick-and-Place (pickplace) · p. 12, 4) Drawer Opening (drawer) · p. 12
• The guidance schedule is uniquely determined by inference delay d and ramp length r, with experiments controlling delay explicitly.Main experiments use d = 8 timesteps, while the delay ablation evaluates d ∈ {6, 8, 10}.Source: B. Delay Construction Details · p. 12

Task completion scoring

• Task completion scores provide graded feedback by rewarding meaningful intermediate steps and penalizing execution errors.Task-specific criteria and penalty rules are defined for all five tasks rather than relying only on binary success.Source: 1) Task Completion Score · p. 13, Table A.1 · p. 13

Smoothness metrics

• Three complementary smoothness metrics—NSPARC, NLDLJ, and overlap RMSE—are reported so that smaller values indicate smoother trajectories.NSPARC measures frequency-domain spectral smoothness, NLDLJ penalizes rapid acceleration changes, and overlap RMSE measures inter-chunk consistency.Source: 2) Smoothness Metrics · p. 13

• Legato improves time, smoothness, and success rates on the Skewer Fruit and Insert Pen tasks.Both tasks require fine manipulation, making them sensitive to jitter; evaluations used 20 trials per method.Source: E. Results on More Challenging Tasks · p. 14

Smoothness–reactivity trade-off

• Legato achieves similar reactivity to RTC while significantly improving smoothness in a reactive ball-pulling task.The task pulls the ball perpendicular to the gripper’s motion at fixed speed, enabling a direct comparison of reactivity and smoothness.Source: F. Experiment on the… · p. 14

Guidance ablation

• Legato significantly outperforms one-shot guidance, particularly on overlap RMSE, under matched pour-task conditions.The comparison uses stride s = 30, delay d = 8, and ramp length r = 22.Source: G. Preliminary Study Details · p. 14
• Repeated guidance before every denoising step is necessary to maintain consistent continuation across action chunks.Initialization-only guidance cannot reliably preserve constraints throughout denoising.Source: G. Preliminary Study Details · p. 14

Trajectory behavior

• RTC exhibits low-frequency, large-amplitude oscillations and more spurious multimodal switching, whereas Legato produces smoother pour trajectories.The trajectory example shows direction changes occurring mostly within RTC chunks despite lower overlap RMSE.Source: Figure A.3 · p. 15

Robot Hardware Configuration

• The platform has two arms, each with seven actuated joints and a gripper, for eight degrees of freedom per arm.Source: H. Robot Hardware Configuration · p. 15
• Three RGB cameras provide visual observations: one head-mounted global camera and one wrist-mounted camera per arm.Source: H. Robot Hardware Configuration · p. 15
• Each arm uses a 7-dimensional end-effector action, giving the dual-arm system a 14-dimensional action space.The action combines a 6-dimensional pose with a 1-dimensional gripper command per arm.Source: H. Robot Hardware Configuration · p. 15

Inference-frequency analysis

• When delay d is fixed, setting s=r=d gives the highest possible inference frequency because the next inference starts immediately after each chunk.Source: 1) Analysis of the d=s=r=8 setting · p. 15

Trajectory smoothness analysis

• RTC shows large-amplitude oscillations in the highest-frequency regime, while Legato produces substantially smoother motion and suppresses low-frequency oscillatory behavior.Legato’s stronger mode persistence across denoising steps is associated with this suppression in the reported analysis.Source: 1) Analysis of the d=s=r=8 setting · p. 15

Metric limitation

• Overlap RMSE can miss smoothness degradation when trajectories contain low-frequency, large-amplitude oscillations.RTC achieves lower overlap RMSE than Legato in the analyzed configuration despite visibly worse oscillatory motion.Source: 1) Analysis of the d=s=r=8 setting · p. 15

Condition-row analysis

• Adding the condition row consistently reduces overlap RMSE but does not significantly improve NSPARC, indicating stronger inter-chunk consistency than intra-chunk smoothness.Smaller delays can also reduce overlap RMSE without the condition row, and omitting it may remain viable when d is sufficiently small.Source: 2) Analysis of the condition row · p. 15