AcrossWAM1.0:A Modular Latent World-Action Stack for Compact Robot Policies

Yafei Zhang; Nan Wu

Source-linked AI summary

AcrossWAM1.0:A Modular Latent World-Action Stack for Compact Robot Policies

Yafei Zhang, Nan Wu

arXiv:2608.29937v1Aug 30, 2026cs.AI

TL;DR

AcrossWAM1.0 addresses the tight coupling among latent world-action components by defining an explicit, auditable modular stack. It evaluates compact scaling on paired LIBERO episodes, finding near-retained performance with fewer deployable parameters while leaving frozen cross-family transfer unresolved.

Problem
LaWAM left the world model, multimodal backbone, and deployment checkpoint tightly coupled, motivating a modular and auditable deployable policy stack.
Method
AcrossWAM1.0 isolates backbone processing with adapters, defines fixed world-action interfaces and hidden-size alignment, separates training-only modules, and verifies an inference-reachable export.
Results
97.45% success versus 98.00% for Qwen3-VL-2B on 2,000 paired LIBERO episodes, a 0.55 percentage-point decrease with exact McNemar p = 0.2664, while unique inference-reachable parameters decrease by 42.4%.
Takeaways & Limitations
The evidence supports compact scaling within the Qwen family and an auditable deployment boundary for latent world-action policies.
Takeaways & Limitations
Closed-loop evidence covers only Qwen-family scaling on LIBERO; frozen cross-family performance-preserving portability is not established, and each backbone uses one trained checkpoint.

Abstract

from arXiv · show

Latent world-action models avoid rendering future pixels by predicting an action-relevant visual subgoal in feature space. LaWAM established this formulation, but its original presentation left the world model, multimodal backbone, and deployment checkpoint tightly coupled. We introduce AcrossWAM1.0, a modularization and scaling study of this latent world-action stack. Rather than presenting latent subgoals as a new algorithm, we make the module boundary explicit: a policy adapter produces latent-action and action-generation contexts; a retained latent world decoder grounds the predicted transition in the current scene;and a flow-matching expert generates continuous action chunks. We further separate training-only teachers from the inference graph and provide a verifiable deployment export. On 2,000 paired LIBERO episodes, replacing a Qwen3-VL-2B backbone with Qwen3.5-0.8B yields 97.45% success versus 98.00% for the 2B model (a-0.55percentage-point difference; exact McNemarp=0.266). This does not prove equivalence, but it meets a prespecified two-point retention criterion. The compact, inference-reachable checkpoint contains 1,472.6M unique parameters, 42.4% fewer than the original 2B policy, while all retained tensors are bitwise identical to the source checkpoint. Cross-family execution is additionally checked with a MiniCPM-V adapter smoke test; closed-loop cross-family transfer remains an open evaluation. AcrossWAM1.0 therefore contributes an auditable software and evaluation boundary for compact latent world-action policies, distinct from LaWAM's original latent-subgoal contribution.

1 INTRODUCTION

AcrossWAM1.0 turns LaWAM’s latent world–action computation into an explicit, auditable policy stack. It isolates backbone-specific processing, separates training-only components from deployment modules, and evaluates compact scaling under matched closed-loop conditions.

Evaluation: The compact study compares full and compact backbones on identical closed-loop episodes rather than treating modularity as a diagram-level claim.The evaluation is designed to test whether behavior is retained after backbone scaling and whether the exported checkpoint removes training-only state.
Deployment boundary: Training-only teachers and paths are excluded from the inference-reachable deployment graph.The compact experiment freezes the Stage-1 teacher and DINOv3 while fine-tuning the retained latent world decoder.
System boundary: AcrossWAM1.0 makes the policy–world–action boundary explicit around LaWAM’s inherited latent world–action computation.The stack exposes adapter outputs, latent decoding, and flow-conditioned action generation as separate components.
Modularization: The backbone adapter returns separate transition and action contexts while hidden-size alignment, query ownership, and gradient boundaries are specified.These contracts address backbone-specific token conventions, dimensions, image-call signatures, and vocabulary handling.
Compatibility: AcrossWAM1.0 retains the legacy LaWAMFramework registry name for checkpoint compatibility while reserving ACROSSWAM1.0 for the modular stack and new experiments.

2 RELATED WORK

Related work supplies semantic and motor priors, latent-action abstractions, and explicit future-state prediction. AcrossWAM1.0 positions itself as a reusable systems and evaluation stack that makes LaWAM’s module boundary auditable under scaling and deployment.

Vision–language–action policies: VLA backbones provide semantic representations, while action-chunking and flow-matching methods provide temporally coherent continuous control.These approaches do not necessarily mediate action generation through an explicit prediction of the action-induced future state.
Latent actions from video: Latent-action methods learn compact transition variables or transferable action abstractions, whereas AcrossWAM1.0 emphasizes decoding the latent action into a policy-relevant visual subgoal.
World models for robot control: World-action models couple action prediction with future-state generation, while LaWAM avoids pixel reconstruction by using a latent visual subgoal.AcrossWAM1.0 builds on that latent-prediction result by exposing the policy/world boundary and evaluating a compact configuration.
From an algorithm to a reusable stack: AcrossWAM1.0 shifts the question from whether latent visual subgoals work to whether their module boundaries survive backbone scaling and deployment export.Its evidence is stronger than a diagram-level claim because it combines matched closed-loop evaluation with tensor-level checkpoint verification.

3 METHOD

AcrossWAM1.0 makes the latent world–action stack modular and auditable by isolating backbone processing, defining fixed query interfaces, and separating training-only components from inference. Its policy predicts a latent action, decodes a grounded visual subgoal with a retained world model, and generates continuous action chunks through a flow-matching expert.

Problem formulation: The model factorizes control into a latent action, a latent visual subgoal, and an action expert conditioned on current context and the predicted transition.The latent visual subgoal makes the predicted scene change an explicit intermediate variable rather than an implicit policy representation.
Stage 1: latent world model: Frozen DINOv3 features provide the current and future targets, while the inverse-dynamics Q-Former encodes their transition into a low-dimensional latent.The released configuration uses dz = 32 and one query; freezing the feature encoder stabilizes the target space.
Stage 1: latent world model: The latent world decoder receives current feature tokens and the latent action, using adaptive LayerNorm modulation to predict a spatially structured future feature.The predicted subgoal preserves the current feature’s spatial token structure and grounds changes in the scene.
Stage 2: policy and action generation: The policy adapter assigns eight latent-action queries to transition inference and eight flow queries to action generation, with a mapper producing the policy latent.The query layout gives latent-action queries a specific dynamic meaning instead of allowing arbitrary action-head features.
Stage 2: policy and action generation: The flow-matching action expert fuses backbone context with current and predicted-future visual tokens, then integrates a learned vector field into an action chunk.The released configuration predicts H = 50 actions over 1.2 s using 10 integration steps.
Inference and deployment: Inference uses only the current observation and instruction, while the inverse-dynamics encoder, state-grounding head, and future frame remain outside the inference graph.The deployment boundary removes training-only teachers and future observations from runtime computation.
Architecture and boundaries: AcrossWAM1.0 defines a policy–world–action contract with backbone adapters, hidden-size alignment, query ownership, and explicit training–deployment boundaries.The reported design also distinguishes interface compatibility, closed-loop scaling, and frozen cross-family transfer.

4 EXPERIMENTS

AcrossWAM1.0 evaluates compact-backbone scaling, deployment export, latent prediction visualization, and adapter portability within its modular latent world–action stack. The experiments show retained LIBERO performance and auditable parameter removal, while cross-family validation remains limited to execution-level testing.

Experimental setup: 2,000 paired LIBERO episodes use identical suite, task, episode, seed, task-order, and initialization keys for both policies.The benchmark covers long-horizon, object, goal, and spatial suites with 500 episodes per suite.
Recent benchmark context: 97.45% LIBERO success is achieved by AcrossWAM1.0 on standard LIBERO with a 1,472.6M-parameter inference-reachable checkpoint.The result is reported as compactness evidence rather than a state-of-the-art claim.
Compact-backbone study: 0.55 percentage points separates the compact model from the 2B policy, satisfying the prespecified two-point retention criterion.The paired comparison used 2,000 episodes, with exact McNemar p = 0.2664 and 1,949 successful closed-loop episodes.
Qualitative latent prediction visualization: Figure 2 compares simulator observations, encoded ground-truth latents, predicted latents, and latent-change magnitude across rollout steps 0, 11, 22, and 33.Shorter horizons preserve scene layout and task-relevant motion, while blur accumulates during long autoregressive rollout; the decoder is visualization-only.
Deployment export: The export retains inference-reachable state while removing the inverse-dynamics teacher, state decoder, unused quantization state, and duplicate aliases.Verification reports bitwise-identical retained tensors, deliberately removed training-only keys, and reproducible alias keys.
Adapter contract validation: MiniCPM-V adapter testing verifies a shared execution contract but does not establish cross-family manipulation performance.The smoke test combines inputs_embeds and pixel_values, supports injected queries, aligns hidden size automatically, and shows decreasing training loss.
Language grounding diagnostic: The language diagnostic reports a trade-off: stronger guidance improves conflicting-instruction adherence but increases collapse on non-conflicting instructions.The diagnostic is secondary and is not treated as the core AcrossWAM1.0 contribution.

5 LIMITATIONS

AcrossWAM1.0’s evidence is bounded to two Qwen-family backbones and LIBERO for closed-loop evaluation, while MiniCPM-V is tested only for execution compatibility. Additional limitations include missing deployment-system measurements, single checkpoints per backbone, and a language adherence–completion trade-off.

Scope of evidence: Closed-loop evidence covers two Qwen-family sizes and LIBERO only; MiniCPM-V receives execution-contract testing rather than control evaluation.Cross-family transfer, frozen-decoder transfer, and broader backbone portability remain unestablished.
Evaluation boundaries: One trained checkpoint per backbone does not measure training-seed variation, and compact-model latency, peak memory, and hardware validation are missing.Episode pairing controls evaluation stochasticity but not training variability.
Language diagnostic: The left/right diagnostic covers only two RoboTwin tasks and exposes an adherence–completion trade-off that should not be generalized to language grounding overall.The limitation concerns both the narrow diagnostic scope and its observed behavioral trade-off.

6 CONCLUSION

AcrossWAM1.0 is an auditable modular stack around LaWAM’s latent world–action computation, contributing explicit interfaces, compact-backbone evaluation, and deployment verification. Its research use remains bounded by physical-world safety responsibilities and reproducibility artifacts.

Conclusion: AcrossWAM1.0 contributes an auditable modular stack around LaWAM’s latent world–action computation.Its contribution is distinguished from LaWAM’s original latent visual-subgoal formulation.
Safety: Physical deployment can cause collisions or damage under out-of-distribution scenes, ambiguous language, or sensor failures.Recommended safeguards include workspace limits, collision monitoring, emergency stops, and human supervision.
Safety: Downstream users remain responsible for dataset licenses, privacy conditions, and safe operating procedures.
Reproducibility: The accompanying repository provides model code, configurations, adapters, training scripts, evaluators, paired tests, and deployment-checkpoint verification.Machine-readable paired and export-verification results accompany the code.

A IMPLEMENTATION DETAILS

The compact Qwen configuration specifies the backbone, latent-action and flow-query dimensions, action horizon, and flow-expert architecture alongside training hyperparameters.

Model configuration: The compact Qwen configuration uses a 1,024-D backbone hidden state, 32-D latent action, eight latent-action queries, and eight flow queries.It also specifies a 50-step action horizon and 1.2 s physical chunk duration.
Action expert: The configuration uses 10 flow integration steps and a DiT action expert with hidden size 1,024, 16 attention heads, and 16 layers.
Training: Stage-two training uses λp = λd = 0.1, repeated noise sampling twice per batch, and gradient clipping at 1.0.
Training: Stage-two AdamW training uses peak learning rates of 10−4 for the VLM and flow expert and 3 × 10−4 for the latent component.

B COMPACT LIBERO RESULTS BY SUITE

Table 4 reports paired backbone results separately for each LIBERO suite, with 500 episodes per suite.

Per-suite results: Table 4 organizes the compact-backbone comparison by LIBERO suite, using 500 paired episodes per suite.

C EVIDENCE TAXONOMY AND EXPLICIT NON-CLAIMS

AcrossWAM1.0 distinguishes verified evidence levels from explicit non-claims across closed-loop scaling, deployment export, and adapter execution.

Verified evidence: Closed-loop scaling is verified for Qwen3-VL-2B versus Qwen3.5-0.8B on LIBERO.
Verified evidence: Exact deployment export is verified at the state-dictionary and CPU-loader levels.
Verified evidence: Cross-family adapter execution is verified by a MiniCPM-V forward/backward smoke test.
Explicit non-claims: The paper does not claim closed-loop MiniCPM-V or cross-family AcrossWAM1.0 performance.
Explicit non-claims: The paper also does not claim statistical equivalence across training seeds, frozen decoder portability across backbone families, or compact-model latency or peak-memory improvement.