LeapBot-JAC: Joint Action Chains for World Action Models

1 INTRODUCTION

• JAC assigns sparse reference and dense execution roles within a fixed action-token budget, using directly supervised robot controls rather than semantic subgoals or executable waypoints.The reference chain conditions the jointly generated execution chain.Source: §1 · pp. 1–2
• JAC jointly denoises both control chains in one shared expert, aligns their physical-time positions, and lets execution read references subject to host visibility constraints.Only the execution prefix is sent to the robot, and the host world-modeling process is preserved.Source: §1 · p. 2
• The paper distinguishes JAC from prior joint intent-and-action generation by emphasizing directly supervised control references inside an existing expert at a fixed token budget.It evaluates task success across two hosts and studies information-flow direction, reference weight, and reference stride.Source: §1 · p. 2
• 98.3% average LIBERO success for both FastWAM and LeapBot-WA, with LIBERO-Plus totals improving by 16.9 and 4.2 percentage points, respectively.The introduction presents these as evidence across two hosts; later evaluations include additional benchmarks.Source: §1 · p. 2

2 RELATED WORK

• JAC differs from hierarchical approaches by allocating reference and execution roles inside one existing action expert, rather than separating long-term plans from local execution.The paper contrasts this design with methods that jointly generate subgoals and trajectories or progressively densify keyframes.Source: §2 · p. 2
• Unlike video- or latent-based intermediate representations, JAC directly supervises its reference chain with robot controls while retaining the host world process.The related-work discussion contrasts JAC with visual subgoals, embodied reasoning, and learned latent intent.Source: §2 · p. 3
• Other world-modeling work addresses semantic, spatial, or historical conditioning, whereas JAC adds supervision on intermediate robot controls.JAC preserves the host world objective and sampling process while changing action-target organization and visibility.Source: §2 · p. 3
• JAC uses independent noise realizations at a common action noise level, unlike Diffusion Forcing’s independent noise levels across sequence tokens.This comparison concerns the noise schedule, not the action-role allocation itself.Source: §2 · p. 3

3.1 PROBLEM AND FIXED-BUDGET FORMULATION

• JAC formulates target allocation, physical-time positions, cross-chain visibility, and reference supervision as changes within one action expert while retaining the host world process.Conditions include observations, instructions, proprioception, and host-permitted history.Source: §3.1 · p. 3

3.2 TARGET CONSTRUCTION AND BOUNDARY PADDING

• With 32 tokens, the default allocates 16 to references sampled every three controls and 16 to execution controls 0–15; the executed prefix is K=10 on LIBERO.On a fully valid window, the chains cover 26 distinct control times; boundary padding clamps out-of-episode indices and excludes them from direct supervision while keeping them attention-visible.Source: §3.2 · pp. 3–4

3.3 SHARED GENERATION AND TIME ALIGNMENT

• The shared expert jointly generates both chains, aligning positions by physical control time and using independent noise samples at a common noise level.The host’s frequency table is retained for RoPE-based hosts; physical durations must be aligned when control frequencies differ.Source: Figure 2 · p. 4, §3.3 · pp. 4–5

3.4 DIRECTED VISIBILITY AND TRAINING OBJECTIVE

• The role mask permits execution queries to read references but blocks direct reverse access, while the host’s other visibility restrictions remain in force.The objective combines execution and weighted reference losses with the unchanged host world loss; directionality does not itself imply statistical independence.Source: §3.4 · p. 5

3.5 INFERENCE AND RESOURCE ACCOUNTING

• At inference, both chains are generated, but only the execution chain’s first K controls are returned and executed before replanning.At fixed token budget, JAC preserves action parameter count and action-expert forward counts while retaining host world updates; structural equality alone does not establish equal runtime.Source: §3.5 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation covers two WAM hosts, LIBERO and zero-shot LIBERO-Plus, RoboTwin, and two physical tasks; local Base/JAC comparisons hold the action-token count at 32.LIBERO-Plus uses no Plus training, finetuning, or test-time parameter adaptation; RoboTwin comparisons use K = 16.Source: §4 · p. 6, §4.1 · p. 6

4.2 MAIN RESULTS

• LIBERO average success reaches 98.3 for FastWAM (from 97.0) and LeapBot-WA (from 97.3); LIBERO-Plus Total rises 53.5 to 70.4 and 73.1 to 77.3.LIBERO-Plus evaluations use policies trained on LIBERO, without Plus training, finetuning, or test-time parameter adaptation.Source: §4.2 · pp. 6–7, §4.1 · p. 6, Table 1 · p. 7
• RoboTwin Avg. rises from 91.02 to 91.24 on FastWAM and from 91.49 to 92.00 on LeapBot-WA.These comparisons execute the first 16 predicted actions from each chunk; FastWAM gains on Random but declines on Clean.Source: §4.2 · p. 6, Table 1 · p. 7

4.3 ABLATION STUDIES ON LIBERO

• Directed R→A visibility scores 98.3, versus 98.2 for reverse access, 97.7 with no direct cross-chain access, and 97.5 bidirectionally.This is the best tested visibility pattern and becomes the default design.Source: Table 3 · p. 8, §4.3 · p. 8
• At s = 2, reference weights 0.1, 0.2, 0.5, and 1.0 yield 97.8, 98.1, 97.5, and 96.2; the moderate weight performs best.The no-JAC FastWAM baseline is shown separately, not included in the weight sweep.Source: Figure 4 · p. 8, §4.3 · p. 8
• At λR = 0.2, strides 2, 3, and 4 yield 98.1, 98.3, and 98.0, respectively, all above the 97.0 FastWAM baseline.Stride 3 is best; the observed range is 0.3 percentage points.Source: Figure 4 · p. 8, §4.3 · p. 8

4.4 COMPUTATIONAL COST

• JAC reallocates 32 action tokens as 16R + 16A without adding parameters or denoising steps, with essentially unchanged latency and peak memory.The computational-cost measurements use model-call latency and peak allocated GPU memory.Source: §4.4 · p. 8, Table 4 · p. 9

4.5 REAL-WORLD EVALUATION

• Across two real-world tasks, JAC raises success on both hosts: FastWAM reaches 93%/79% and LeapBot-WA 95%/90% on sorting/drawer closing.Each model has 200 trials per task and one training seed.Source: §4.5 · p. 9, Figure 5 · p. 9

• Reference stride and λR jointly affect multiple training factors, limiting causal isolation in the current ablations.Source: §5 · p. 9
• Because sparse reference controls are not explicitly constrained to encode semantic intent, their interpretation and broader transfer remain for future study.Source: §5 · p. 9

6 CONCLUSION

• LeapBot-JAC improves performance across LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world tasks, with the largest gains under zero-shot perturbations.Ablations validate the main design choices, while matched inference shows negligible overhead.Source: §6 · p. 10

A.1 CONFIGURATIONS AND TARGET SPANS

• All local models use one training seed; RoboTwin models are re-evaluated with replan=16, while literature rows retain their published protocols.RoboTwin uses clean and randomized demonstrations.Source: §A.1 · p. 14, Table 5 · p. 14

A.2 REFERENCE-STRIDE ELIGIBILITY

• Across strides 2, 3, and 4, fully available labels yield 24, 26, and 28 distinct target times, while reference span, overlap, and boundary padding can change together.Stride conditions use identical trajectory starts, and padding frequency was not quantified.Source: §A.2 · p. 14

A.3 EXPERIMENTAL SETTINGS

• Host-specific optimization, training schedules, and baseline hyperparameters follow FastWAM and LeapBot-WA, while single-seed results do not characterize training-seed variability.Source: §A.3 · p. 14

A.4 PADDING IMPLEMENTATION

• Out-of-range future indices are clamped to the final legal action, with LIBERO's padded delta components zeroed before the normal action transformations.The nondelta gripper component retains its final legal value, and normalized padded delta components need not be numerically zero.Source: §A.4 · p. 14
• The default 48-control window selects reference indices 0, 3, …, 45 and execution indices 0, …, 15; padding slots undergo the forward pass but are loss-masked.The padding mask is not an attention key-padding mask, so padded references can still influence valid predictions.Source: §A.4 · p. 15

A.5 LOSS ACCOUNTING

• Each chain's loss averages squared flow error over action dimensions and valid tokens, applies the shared timestep weight, and averages over the batch.The inherited scheduler uses a Gaussian-shaped weight centered at the midpoint of its training timestep range.Source: §A.5 · p. 15
• λR weights the normalized reference objective relative to execution, but equal valid-token averaging does not ensure equal loss magnitudes or gradient norms.Padding can change supervised physical times and influence predictions through attention-visible tokens; λR also changes the action objective relative to the world objective.Source: §A.5 · p. 15
• No loss-reweighted single-chain control or zero-weight JAC result is reported, so the base/JAC gain reflects combined design changes and the sweep does not establish direct reference supervision's necessity.Source: §A.5 · pp. 15–16

A.6 METRICS AND RUN IDENTITY

• All local results use a single common training seed, and reported differences are descriptive rather than significance estimates.Real-world rates average four equal-size rounds of 50 trials per model per task; the rounds are not independent training seeds.Source: §A.6 · p. 16
• LIBERO-Plus Total aggregates the full test evaluation and is not reconstructed by averaging the seven displayed categories.Category counts need not be equal, so Total is not a macro-average of those categories.Source: §A.6 · p. 16
• The FastWAM + JAC aggregate of 98.3/70.4 at s = 3 and λR = 0.2 appears consistently in related visibility and stride rows, not as an independent replication.The s = 2, λR = 0.2 result remains 98.1/63.0; displayed precision does not support conclusions about seed variability.Source: §A.6 · p. 16

A.7 ROBOTWIN EVALUATION PROTOCOL

• FastWAM’s Clean/Random/Avg. rates are 91.44/90.60/91.02, versus 91.32/91.16/91.24 with JAC; LeapBot-WA’s are 90.92/92.06/91.49, versus 91.46/92.54/92.00 with JAC.All four local models use clean and randomized demonstrations and replan = 16.Source: §A.7 · p. 16

A.8 HOST INTEGRATION

• The local semantic-cache implementation freezes the V-JEPA encoder and adapter while training the action expert, proprioception encoder, and JEPA context projector.This description is specific to the local implementation and is not assumed to specify every host.Source: §A.8 · p. 16
• Across both hosts and reference strides s = 2, 3, 4, video sampling, world-prediction steps, and world-loss coefficients remain fixed, so longer references add no world targets.The data pipeline retains each host’s baseline video-sampling grid when extending the future-action window.Source: §A.8 · p. 16
• In hosts with feedback through world tokens or caches, direct role masking does not guarantee global independence; reported model-call latency excludes a complete observation-to-command cycle.Action-token counts alone do not determine runtime.Source: §A.8 · p. 16

A.9 FORWARD-DEPENDENCY ANALYSIS

• Under execution-independent context and no reverse path from execution initialization, induction over layers and solver updates gives the stated forward-dependency result.The claim is conditional computation, not statistical independence: the chains share conditions and parameters, and execution losses can backpropagate through reference representations.Source: §A.9 · p. 16
• In coupled hosts, an A → W → R path may violate the condition; removing feedback or copying context would change the host and require separate cost reporting.Source: §A.9 · p. 17

A.10 SAMPLING PROCEDURE

• JAC jointly updates both chains within the existing action expert, follows the host schedule for world-state and cache updates, and sends only the execution prefix to the environment.Sampling initializes host world states and history, independently samples R and A, maps physical-time positions, then executes the first K controls from A.Source: §A.10 · p. 17

B LIMITS OF MECHANISTIC INTERPRETATION

• The three ablations test direct visibility, positive reference-loss weights, and temporal strides, but do not establish reference semantics or abstract planning.Sparse controls may instead act as auxiliary predictive targets, and blocked direct access may leave indirect communication; matched controls are needed to isolate these mechanisms.Source: B LIMITS OF MECHANISTIC INTERPRETATION · p. 17