PaperlayerView exact revision on arXiv

Source-linked AI summary

MASS: Multiplayer World Models with Authoritative Shared State

Ziqi Cai, Siqi Yang, Yimu Wang, Zixian Gao, Yunheng Liu, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi

arXiv:2608.06257v2cs.CVcs.HC

TL;DR

Multiplayer world models lack a typed authoritative state that separates shared world dynamics from camera-dependent rendering. MASS introduces this separation and achieves 0.764 parser recovery versus 0.128 for the strongest video-based baseline while advancing 1,024 players for 10,000 ticks.

  • Problem

    Existing multiplayer world models lack a typed authoritative state shared as recurrent memory and a synchronization reference across camera views.

  • Method

    MASS advances a global typed state from joint actions with a Logic Engine, then uses a Rendering Engine to generate camera-conditioned views.

  • Results

    0.764 parser recovery versus 0.128 for the strongest video-based baseline, with zero measured cross-view disagreement and rollouts of 1,024 players for 10,000 ticks.

  • Takeaways & Limitations

    Explicit authoritative state modeling provides a practical foundation for scalable and consistent multi-agent world simulation.

Abstract

from arXiv · show

Current video world models struggle in multiplayer environments because they entangle world state with view-dependent visual latents, leading to redundant compute, view inconsistencies, and poor scalability. We propose MASS (Multiplayer world models with Authoritative Shared State) to resolve this limitation. Inspired by multiplayer game architectures, MASS disentangles world dynamics and view rendering. A learned Logic Engine advances a global, authoritative typed state from joint actions without any hand-written transition function, acting as the sole recurrent memory and synchronization reference. From this shared state, a learned Rendering Engine generates independent and consistent views for any requested camera on demand. This explicit disentangling allows MASS to achieve superior state accuracy and lower cross-view inconsistency compared to state-of-the-art multi-view baselines on a matched multiplayer Snake benchmark. It advances predicted worlds with 1,024 concurrent players for 10,000 recurrent steps. Our results show that explicit, authoritative state modeling provides a practical foundation for scalable and consistent multi-agent world simulation.

1 Introduction

MASS addresses multiplayer world-model limitations by separating an authoritative typed world state from camera-conditioned rendering. It learns synchronized state advancement and independent views, achieving stronger Snake state recovery and scaling to long rollouts with 1,024 simulated players.

  • Motivation: Multiplayer simulation requires recurrent state belonging to the world rather than any camera, because separate visual histories duplicate shared content and can make views disagree.A thousand players may inhabit one world while each camera reveals only a small part of it.
  • Approach: MASS predicts an authoritative typed state from joint actions, then renders each requested observation conditioned on that state and the client camera.The same predicted state serves as recurrent history, synchronization reference, and source of every client view.
  • Approach: MASS uses a schema-based architecture in which a Logic Engine advances typed entity records and a Rendering Engine generates requested observations without a native game transition.During update gaps, the Logic Engine can temporarily predict from a synchronized client state.
  • Results: 0.76 state recovery on matched Snake compared with 0.128 for the strongest video-based baseline demonstrates MASS’s quantitative advantage.The evaluation includes matched multiplayer baselines, cross-game studies, simulated update stalls, and long recurrent rollouts.
  • Results: 1,024 simulated player entities were advanced for 10,000 recurrent ticks, demonstrating the scale and duration of MASS rollouts.These experiments support the paper’s evaluation of long recurrent rollouts.

2 Related work

Related work spans pixel- or latent-carried video world models, multiplayer architectures that coordinate agents through shared representations or attention, and structured dynamics models. MASS follows authoritative networked-game execution by separating one canonical learned state update from per-camera rendering.

  • Video world models: Video world models recurrently predict action-conditioned observations, using pixels or visual latents as the recurrent carrier for interactive environments.Examples include GameNGen, Oasis, DIAMOND, Genie, and WHAM.
  • Multiplayer generation: Multiplayer-generation methods coordinate agents through global visual representations, sparse hub attention, persistent external memories, cross-agent attention, state alignment, or joint denoising.The passage names Gamma-World, MultiWorld, MultiGen, ShareVerse, MetaWorld, and Prisma-World as examples.
  • Structured dynamics: Structured-dynamics approaches learn compact or recurrent state spaces beyond raw pixels, but their dense or latent representations lack explicit entity structure.The passage discusses learned compact latents, recurrent state-space predictors, neural cellular automata, and convolutional recurrent models.
  • Multiplayer execution: Networked games advance one authoritative canonical state, replicate versioned snapshots, and let each client render its own view; MASS applies this separation to learned world models.MASS runs the learned transition once per tick and serves any number of cameras from the single predicted state.

3 Method

MASS separates learned world dynamics from view rendering by using an explicit typed world state as the sole recurrent and synchronization object. A Logic Engine advances this shared state from joint actions, while independent Rendering Engines generate camera-specific views on demand.

  • Architecture: MASS uses a predicted typed world state as its single recurrent object, separating state transitions from camera-local frame generation.The World State Tokenizer encodes schema-defined records, the Logic Engine advances them, and the Rendering Engine converts projected state into frames.
  • World State Representation: A game schema declares entity kinds, fields, allowed values, and instance counts; the learned model supplies the transition while preserving typed validity.Records use fixed-position type tags so numbers, coordinates, flags, optional values, and lists decode back into fields without loss.
  • Logic Engine: The Logic Engine is the only learned component that advances shared state, using a decoder-only Transformer with record-wise prediction and neighborhood-based interactions.Self-attention is confined within each record, while interactions enter through neighborhood windows computed from the shared state before prediction; this supports 1,024-entity worlds.
  • Rendering Engine: Each requested camera reads the same projected typed state, enabling independent views whose appearance can change without altering or retraining the learned dynamics.The learned residual U-Net renders RGB frames from camera-local projections, and cameras, resolution, materials, and lighting can change at rollout time.
  • Authoritative Shared State: The authoritative server publishes a versioned typed snapshot after each global transition, so clients render the same predicted world while distribution scales linearly in C.The learned transition runs once per tick, rendering runs locally on clients, and the latest authoritative state can initialize temporary client-side prediction during stalls.

4 Experiments

Experiments evaluate MASS under matched multiplayer conditions, across games, scales, and client-update failures. MASS achieves stronger perceptual and state recovery results, while its typed authoritative state supports scalable rendering, error localization, and robust local prediction.

  • Matched Snake benchmark: The matched benchmark fixes held-out episodes, synchronized cameras, initial worlds, resolution, rollout length, data, actions, and optimization budgets across recurrent carriers.Each method is evaluated at 128 × 128 resolution over 128 ticks from the same initial world.
  • Matched Snake benchmark: MASS leads six of seven matched Snake metrics, with LPIPS 0.098 versus 0.123 for B-UN and 0.277 for MultiWorld.Parser recovery reaches 0.764 versus 0.128 for B-PV, while shared typed-state decoding guarantees state-level cross-view consistency.
  • Cross-game generalization: Across eight games, PSNR ranges from 23.72 dB for Tron to 40.24 dB for Frogger, with five games exceeding 32 dB.The same Logic Engine and Rendering Engine architecture uses game-specific weights and declarative schemas, while object PSNR stays above 27.5 dB across most games.
  • Scalability: MASS advances worlds with 1,024 simulated player entities for 10,000 recurrent ticks, while transition cost remains independent of rendered-view count.State distribution and requested rendering are separate workloads because the learned transition runs once per tick.
  • Ablations and error localization: Replacing the typed carrier with a dense joint predictor reduces parser recovery to zero and nearly eliminates recoverable interaction events, despite LPIPS remaining 0.123.The dense grid fails to preserve entity identity across ticks, whereas the explicit state interface separately reveals dynamics and rendering errors through SRSC and reference-state replacement.
  • Missing server updates: With stalled server updates, the local avatar remains correct at every evaluated stall length, while oracle joint inputs match the server for all eight ticks.Agreement for other visible objects decreases as unknown remote actions accumulate, attributing discrepancies to missing actions rather than the learned transition.

5 Conclusion … A.2 Additional details of MASS

MASS separates learned world dynamics from camera-conditioned rendering through a shared authoritative typed state. Its declarative schemas and shared-state rollout procedure support structurally valid, consistent, and scalable multiplayer simulation across games.

  • 5 Conclusion: MASS uses one shared predicted state to synchronize simultaneous cameras, separate world-transition cost from rendered-view count, and support 1,024 simulated players.The same compact Logic Engine and Rendering Engine also support several games through declarative schemas, long structurally valid rollouts, and client-side prediction.
  • A.1 A real game schema: A single declarative schema specifies game entities, typed fields, spatial anchors, and neighborhood context, while transition outcomes are learned from recorded trajectories.The population-scale Snake schema declares 1,024 snakes and 4,096 spatially bucketed food rows; the Logic Engine receives no hand-written transition rules, rewards, or game code.
  • A.2 Additional details of MASS: The shared world state is MASS’s sole recurrent object: the Logic Engine advances it once, and the Rendering Engine answers camera queries from the resulting state.Every view uses the same predicted world with its own camera and player conditioning, while rendering cost grows with the requested camera set.
  • Appendix: Schema-derived serialization enforces typed vocabularies, canonical ordering, padding, and uniqueness, while movement, collision, growth, collection, death, and reward remain learned outcomes.These structural constraints separate semantic transition accuracy from structural decodability.
  • A.2 Additional details of MASS: MASS assembles all next records before publication and rendering, then feeds the same assembled prediction into recurrence, synchronization, evaluation, and image generation.Every prefix is built from one current predicted state, so each tick contains one world transition before camera-specific rendering.
  • A.2 Additional details of MASS: Client-side prediction initializes the same Logic Engine from the latest versioned authoritative state and locally advances the full typed world when server updates are missing.When a new authoritative version arrives, the client replaces speculative state directly, without a correction network or hidden RGB latent.

B The matched multiplayer benchmark · B.1 Experimental protocol · B.2 Matched Snake codec

The matched benchmark evaluates whether shared recurrent state preserves one recoverable multiplayer world across consistent views, using controlled Snake trajectories, fixed cameras, and semantic state metrics. Its matched codec represents Snake’s canonical world as typed state while leaving gameplay transitions learned.

  • B The matched multiplayer benchmark: The benchmark tests whether a recurrent carrier preserves one recoverable world across two views before rendering and across different world sizes, camera counts, and visual domains.Direct state evaluation, population-scale studies, and cross-game studies assess the same state–camera decomposition.
  • B.1 Experimental protocol: 48 × 48 Snake arenas provide matched multi-agent trajectories for populations N ∈{2, 4, 8}, with episode-separated training, validation, and test splits.Training uses 128 episodes per population; validation and test use 16 and 32 episodes per population, respectively.
  • B.1 Experimental protocol: Every method receives identical recorded states, initial images, joint actions, and player-centered camera paths, including trajectories with collection, growth, collisions, deaths, and interactions.Cameras use a 31-cell viewport, and paths are fixed from recorded reference states rather than recentered from predictions.
  • B.1 Experimental protocol: Five recurrent carriers are compared: MultiWorld, independent-view B-PV, shared-latent B-SL, dense-state B-UN, and MASS.B-UN isolates whether a dense shared carrier can preserve sparse entity identity and position under recurrence.
  • B.1 Experimental protocol: 128-tick test protocols and a shared 20,000-update budget standardize the matched comparison for internal baselines and MASS, while MultiWorld uses its official 35-epoch schedule.All methods use the same training episodes, actions, and camera paths.
  • B.1 Experimental protocol: Parser, Count, Position, Event F1, X-view, Invalid, and direct typed-state metrics separately measure recovered content, cross-view disagreement, structural validity, and exact state correctness.X-view compares parsed semantic classes in overlapping world-coordinate regions, while Full exact requires the complete canonical state to match.
  • B.2 Matched Snake codec: The matched Snake codec uses a 421-token canonical state containing the tick, 64 sorted food coordinates, and ordered per-player body and status fields.A 447-token input prefix adds current state, all player actions, and up to eight net-added food coordinates.
  • B.2 Matched Snake codec: The schema-generated decoder enforces declared types, ranges, ordering, uniqueness, and canonical padding, while gameplay transitions remain learned rather than hand-coded.The mask prevents repeated body coordinates and preserves sorted unique food coordinates; semantic state metrics remain distinct from structural validity.

B.3 Extended results on matched multiplayer Snake

Matched multiplayer Snake results show that MASS preserves recoverable entity state, position, event timing, and cross-view agreement over recurrent rollouts, unlike baselines whose visual plausibility can conceal state failure. Direct evaluation and logic-rendering decomposition further attribute these gains to the typed recurrent state rather than rendering alone.

  • Performance over rollout horizon: 0.963 foreground-state recoverability at tick 1 declines to 0.642 at tick 128 for MASS, while no baseline exceeds 0.19 initially.MultiWorld loses most parsable content by tick 8, whereas B-UN begins near zero despite comparatively low LPIPS.
  • Performance over rollout horizon: 1.000 position recall at tick 1 falls to 0.224 at tick 128 for MASS, while observation-entangled baselines develop persistent cross-view disagreement.B-UN lowers disagreement through a dense shared state, but that state contains little recoverable world content.
  • Performance over rollout horizon: 0.841 cumulative Event F1 at the start remains 0.702 through tick 32 for MASS, while image predictors approach a similar endpoint with world recoverability below 0.15.Joint parser, cross-view, and event scores distinguish event timing from recoverable entities and simultaneous-view agreement.
  • Direct logic evaluation: 99.1% position accuracy with no contradiction at one step, and 90.2% at H = 32, are achieved by the typed model; dense predictors remain below 2.7%.Every dense predictor exceeds 91% semantic accuracy at one step, yet none exceeds 2.7% position accuracy and every predicted tick is contradictory.
  • Logic and rendering decomposition: 95.0 mean renderer-ceiling score on Crate Pusher and 69.0 on Snake reference states show stable rendering when state quality is fixed.The Snake reference-state score has a sample standard deviation of 4.4 points across horizons, isolating recurrent state evolution in the remaining comparisons.

B.4 Logic dynamics · C Scaling and stability studies

MASS’s typed-token logic dynamics preserve semantic, positional, and structural accuracy across long rollouts, outperforming dense-state diagnostics under matched evaluation. Scaling training and changing token order materially affect exactness and head-position accuracy, revealing both gains and trade-offs.

  • B.4 Logic dynamics: Typed-token Transformer evaluation uses identical trajectories, splits, update budgets, rollout initialization, horizons, episodes, and metric code across three dense-state diagnostics.The diagnostics are a 3.26M independent-head CNN, a 3.86M RSSM, and a 3.69M joint U-Net.
  • B.4 Logic dynamics: 97.9% semantic, 94.9% active-field, and 99.1% head-position accuracy are achieved at H = 1, while dense predictors reach at most 2.7% head-position accuracy.The typed configuration separates positional continuity from aggregate field accuracy.
  • B.4 Logic dynamics: 0 structural contradiction rate is maintained by the typed representation, whereas every dense predictor is contradictory on every evaluated tick.This comparison is reported under the matched logic-dynamics evaluation.
  • B.4 Logic dynamics: 90.8% semantic and 90.2% head-position accuracy remain at H = 32 with no collapse, while full-state exact score reaches 41.7% at H = 1.The reported long-horizon results show retained semantic and positional accuracy alongside lower full-state exactness.
  • C Scaling and stability studies: 100,000 updates raise H = 1 exactness to 45.8% and head-position accuracy at H = 32 and H = 128 to 96.4% and 83.0%, respectively.These results come from extending the same-seed budget from 20,000 to 100,000 updates.
  • C Scaling and stability studies: Reordering player tokens before food raises H = 1 exactness from 41.7% to 45.8%, while H = 32 head-position accuracy changes from 90.2% to 80.4%.The token-order intervention improves exactness but reduces the reported H = 32 positional accuracy.

C.1 Cross-game results

Cross-game evaluation changes only the declarative codec and game data under fixed training settings, using matched initial states, action streams, and horizons. Breakout achieves high short-horizon accuracy and retains structural validity at long horizons, while Tile Merger remains structurally valid despite zero full-state exactness.

  • Evaluation setup: The cross-game comparison fixes width 256, six layers, eight heads, batch size 8, 20,000 updates, and the seed, varying only codec and game data.Breakout contributes 3,466 training and 855 validation transitions.
  • Evaluation protocol: Each game uses its first validation window in manifest order, with prediction-reference pairs sharing the initial state, action stream, and horizon.The comparison is independent of visual quality.
  • Cross-game results: 99.5% active-token accuracy and 84.5% full-state exactness are reached by Breakout at H = 1.At H = 128, Breakout records 85.7% active-token accuracy and 3.2% full-state exactness.
  • Cross-game results: 72.5% active-token accuracy and structural validity are retained by Tile Merger at H = 128, while full-state exactness is zero across reported horizons.These results distinguish structural validity from stochastic-grid exactness.

C.2 Rendering across games · C.3 Long-horizon stability

MASS keeps a fixed typed-state and camera contract across Snake, Crate Pusher, and Pac-Man while using separately trained renderers for each visual domain. In long-horizon Snake stress tests, populations retain structural validity and roster integrity through H = 4096, while trajectories exhibit diverse evolution before recurrence.

  • C.2 Rendering across games: The shared state and camera interface transfers across Snake, Crate Pusher, and Pac-Man despite changes in game geometry, object semantics, and appearance.Each game uses population-scale 1,024-entity states with a separately trained renderer.
  • C.2 Rendering across games: For Snake, the two renderers obtain PSNR 33.54 and 30.12 with SSIM 0.9922 and 0.9855 on held-out reference states.These measurements isolate view synthesis from recurrent state prediction.
  • C.2 Rendering across games: For Crate Pusher, the two renderers obtain PSNR 35.38 and 34.39 with SSIM 0.9970 and 0.9898 on held-out reference states.The same state and camera contract is paired with separately trained 2D and 2.5D renderers.
  • C.2 Rendering across games: For Pac-Man, the two renderers obtain PSNR 34.88 and 32.72 with SSIM 0.9952 and 0.9795 on held-out reference states.These measurements isolate view synthesis from recurrent state prediction.
  • C.2 Rendering across games: All six renderers exceed SSIM 0.97 on held-out reference states, separating renderer quality from recurrent state evolution.The interface is shared across games, while renderer weights remain game-specific.
  • C.3 Long-horizon stability: The long-horizon stress test measures structural decodability, roster survival, collapse, and recurrence independently of a reference continuation.Recorded Snake trajectories provide 159 reference transitions, while endpoint measurements and rollout curves report the stress-test behavior.
  • C.3 Long-horizon stability: Before entering a recurrent attractor, trajectories visit between 277 and 306 unique predicted states, complementing structural and collapse metrics.With the 100,000-update checkpoint, two initializations produce 1,850 and 302 unique states.

C.4 Large-scale and asynchronous rollouts · D Netcode and diagnostics · D.1 Client prediction

MASS supports large-scale asynchronous rollouts by advancing one authoritative typed state and serving multiple camera views from it, while client prediction preserves local responsiveness during update gaps. With complete inputs, rollouts remain exact; with missing remote inputs and spawns, local views diverge progressively while the evaluated avatar remains positionally exact.

  • C.4 Large-scale and asynchronous rollouts: The Logic Engine predicts one shared state containing 5,121 records for 1,024 players and 4,096 food buckets per world tick.A controlled H100 renderer sweep queried 1, 4, 16, 64, 256, or 1,024 views from the fixed prediction.
  • C.4 Large-scale and asynchronous rollouts: Increasing requested views changes rendering workload without changing the learned world transition, because all cameras query one prediction.MASS avoids maintaining separate recurrent worlds for separate views.
  • C.4 Large-scale and asynchronous rollouts: With only the evaluated player’s action, the selected avatar remains exact in all 24 windows while unseen actions and events progressively alter the surrounding world.The client assigns no-op actions to other players and withholds future food spawns.
  • C.4 Large-scale and asynchronous rollouts: With complete joint inputs, local rollouts match authoritative in-view state in all 24 asynchronous windows.The windows begin at three held-out anchors and last from one to eight ticks.
  • D.1 Client prediction: Client prediction advances the full typed state from the latest authoritative update during server stalls, using Snake at N = 1024 and anchors t ∈{100, 300, 600}.Each anchor is evaluated for up to k = 8 ticks against the recorded authoritative state.
  • D.1 Client prediction: The oracle joint-input variant maintains in-view agreement 1.000 through k = 8, whereas the client variant decays from 0.815 at k = 1 to 0.429 at k = 8.The decay reflects unknown remote inputs and withheld exogenous spawns rather than learned-transition divergence within the tested window.
  • D Netcode and diagnostics · D.1 Client prediction: Under the evaluated anchors, the local avatar incurs zero displacement, so no position correction is needed for its predicted continuation.Reconciliation corrects the surrounding-world divergence when a new server state arrives.
  • D.1 Client prediction: At the matched two-player scale, the full-state transition takes 45.6 ms and fits a 20 Hz tick budget.The correctness measurement does not vary neighborhood projection, transport policy, or wall-clock optimization.

D.2 SRSC protocol and results

SRSC uses a vision-language judge to evaluate semantic correctness beyond visual similarity, while MASS’s typed-state interface separates logic-transition and rendering errors. Results show horizon-dependent logic degradation, moderate predicted-state rendering scores, and no substantial temporal semantic drift for ground-truth-state rendering.

  • Motivation: SRSC evaluates semantic correctness because visually similar frames can still contain incorrect entities, positions, identities, or interactions.It uses a VLM to judge semantic content rather than relying only on PSNR, SSIM, or LPIPS.
  • Evaluation axes: MASS’s explicit typed state enables separate evaluation of the Logic Engine and Rendering Engine by substituting ground-truth or predicted states and renderers.The Logic axis compares teacher renders of ground-truth and predicted states, while rendering axes hold the state fixed to isolate rendering behavior.
  • Protocol and prompt: Qwen3.5-27B judges symbolic semantic rasters or learned RGB frames by inventorying entities, matching identities and locations, and checking states and interactions.The protocol ignores artistic style, texture, lighting, and sharpness, focusing on semantic content.
  • Results: On Snake, the Logic axis remains above 90% through H=100, Renderer-Pred scores range from 30–77%, and end-to-end scores follow logic-axis decay.Crate Pusher degrades faster with higher interaction density, whereas on Pac-Man Renderer-Pred is 100% at H=1 and 65% at H=300.
  • Results: 63.8–75.0: Renderer-GT scores fluctuate without monotonic decline, suggesting no substantial semantic drift over the evaluated horizons.The remaining offset from the symbolic-raster ceiling reflects the representational gap between categorical rasters and learned RGB rendering.
Loading 2608.06257v2…