PaperlayerView exact revision on arXiv

Source-linked AI summary

Multics: Generative Multi-Player World Model with Hierarchical Communication

Kaiyi Huang, Wangbo Yu, Yukun Huang, Jiwen Yu, Kunhao Liu, Donglin Yang, Wenbo Hu, Ying Shan, Xihui Liu

arXiv:2610.17201v1cs.CV

TL;DR

Multi-player world models must generate consistent, controllable views while scaling to more players, but existing communication approaches face computational or interference limitations. Multics uses hierarchical communication with budget-constrained dynamic grouping. It reports state-of-the-art results in 4- and 6-player settings, generalizes to larger player counts without retraining, and reduces self-attention computation relative to fully connected communication.

  • Problem

    Multi-player world models need robust, consistent player-specific generation that scales to more players, while fully connected communication has quadratic cost and centralized representations can mix unrelated events.

    Source: §1 · p. 2

  • Method

    Multics uses group tokens to aggregate locally relevant player information, connect groups through global tokens, and dynamically group players based on recent camera trajectories under a communication budget.

    Source: §1 · p. 2, §3.3 · p. 6

  • Results

    Multics achieves state-of-the-art performance in 4- and 6-player settings, with strong consistency, controllability, and interference robustness, generalizes to 6-, 8-, and 10-player inference without retraining, and reduces self-attention computation versus fully connected communication.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Shared group-token parameters let the same communication module accommodate varying player counts without introducing new group-specific parameters.

    Source: §1 · p. 2

    The work primarily focuses on cross-player consistency, controllability, and robustness to cross-event interference; maintaining persistent world states across multiple players over long horizons remains future work.

    Source: D LIMITATIONS · p. 18

Abstract

from arXiv · show

Multi-player interactive world models aim to simulate shared environments from multiple players' perspectives, but achieving robust generation while scaling to more players remains challenging. Existing approaches that rely on fully-connected communication (e.g., Solaris) incur quadratic attention costs as the number of players grows, while centralized approaches (e.g., $\gamma$-World) suffer from interference between unrelated events in a shared global representation. To address these challenges, we present Multics, a multi-player video generation framework with hierarchical communication that structures information exchange across player, group, and global levels. This hierarchy introduces group tokens into self-attention as intermediate group-level representations that aggregate locally relevant player information. These tokens connect to global tokens for cross-group communication, while the hierarchical topology limits direct mixing between unrelated local events. Group-token parameters are shared across group slots, allowing the same communication module to accommodate varying numbers of players without introducing new group-specific parameters. We further propose budget-constrained dynamic grouping based on recent camera trajectories, which adaptively groups relevant players while limiting unnecessary communication. Experiments demonstrate that Multics achieves state-of-the-art performance in 4- and 6-player settings, with improved cross-player consistency, player-specific action and camera controllability, and interference robustness. Moreover, Multics generalizes from 4-player training to 6-, 8-, and 10-player inference without retraining, while reducing self-attention computation compared with fully-connected communication.

1 INTRODUCTION

Multics structures player communication hierarchically and uses dynamic grouping to address robust generation and scalability. It reports strong results in 4- and 6-player settings, generalization to larger player counts without retraining, and reduced self-attention computation.

1 INTRODUCTION

  • Multi-player world models must maintain cross-player consistency, player-specific action and camera control, and robustness to unrelated concurrent events while scaling to more players.Fully connected communication has quadratic computational cost as player count grows, while centralized shared representations can mix unrelated events.Source: §1 · p. 2
  • Multics uses group tokens to aggregate locally relevant player information and connect to global tokens for cross-group communication, limiting direct mixing between unrelated local events.Group-token parameters are shared across group slots, so the same module accommodates varying player counts without new group-specific parameters.Source: §1 · p. 2
  • Budget-constrained dynamic grouping uses recent camera trajectories to assign relevant players to groups while limiting unnecessary communication.The approach adapts group assignments to player relevance.Source: §1 · pp. 2–3
  • Multics achieves state-of-the-art performance in 4- and 6-player settings, generalizes from 4-player training to 6-, 8-, and 10-player inference without retraining, and reduces self-attention computation versus fully connected communication.It also reports strong cross-player consistency, controllability, and interference robustness.Source: §1 · pp. 2–3

2 RELATED WORK

Related work spans video generation and world models, several approaches to multi-player communication, and multi-agent communication topologies. Multics studies hierarchical exchange within high-dimensional video representations for multi-player world modeling.

Video Generation and Video World Models

  • Video-generation research spans Diffusion Transformers, token-prediction, and hybrid methods, with conditioning on cameras, actions, and multimodal references supporting controllable generation.Video world models generate future observations from past states and actions and have been explored across robotics, games, driving, and physical simulation.Source: §2 · p. 3

Multi-Player World Models

  • Multi-player world models use dense joint generation, centralized representations, or persistent shared world states; Multics instead structures communication within learned visual representations.Dense methods couple player streams at increasing computational cost, while shared-state approaches primarily address population scalability through state-rendering decomposition.Source: §2 · p. 3

Communication Topologies

  • Multi-agent communication ranges from dense exchange to selective or targeted communication and hierarchical exchange across agent groups; this work studies communication within video representations.The cited approaches primarily support multi-agent decision making.Source: §2 · p. 3

3 METHOD

Multics jointly generates player videos conditioned on player-specific inputs, aiming to preserve cross-player consistency and follow each player’s action and camera controls. It structures communication hierarchically across player, group, and global levels.

3 METHOD

  • The objective is to jointly generate player videos that preserve consistency across overlapping views and follow player-specific action and camera controls.Source: §3 · p. 3

3.1 MULTI-PLAYER VIDEO GENERATION WITH CONDITIONING

  • Each player’s initial image anchors generation; action sequences, camera trajectories, and text prompts condition the corresponding player tokens.Discrete controls and relative yaw/pitch changes enter through separate pathways; camera features are injected into query and key features, while prompts use cross-attention.Source: §3.1 · p. 4

3.2 HIERARCHICAL COMMUNICATION FOR MULTI-PLAYER

  • Solaris uses direct all-player attention, centralized communication uses shared global hubs, and group-based communication restricts direct player links to group members.Source: §3.2 · pp. 4–5
  • Multics inserts group tokens as intermediate representations for locally relevant information; shared parameters across group slots support varying player counts without new group-specific parameters or retraining.The authors attribute improved consistency, player-specific controllability, and interference robustness over communication alternatives to local aggregation before broader global exchange.Source: §3.2 · p. 5

3.3 DYNAMIC GROUPING

  • Dynamic grouping estimates player relevance from recent camera trajectories and selects groups under limits on group slots, group capacity, and communication cost.The selection maximizes player coverage while preserving high-relevance relations; unassigned players use remaining slots and budget or communicate only through global tokens.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

The experiments compare Multics with fully connected and centralized methods, evaluate scaling and cross-domain applicability, and test its communication and grouping designs. Results report stronger multi-player generation alongside reduced computation relative to fully connected communication.

4.1 EXPERIMENT SETTING

  • Experiments use OpenCS2 and EgoTeam, compare reimplemented baselines under matched training configurations, and assess consistency, action and camera accuracy, and visual quality.OpenCS2 models use 165-frame videos at 352×640 resolution; evaluation uses 200 held-out OpenCS2 samples and 100 EgoTeam test samples.Source: §4.1 · p. 6

4.2 COMPARISON WITH PREVIOUS METHODS

  • Multics outperforms Solaris and γ-World on most evaluation metrics in both 4- and 6-player settings, with stronger consistency, controllability, and visual quality.Qualitative comparisons also report stronger interference robustness.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3 SCALABILITY AND GENERALIZATION ABILITY OF Multics

  • With 10 players, Multics uses 9.6 TFLOPs versus 69.1 TFLOPs for fully connected communication, while remaining comparable in efficiency to centralized and group-based methods.Fully connected communication scales quadratically with player count.Source: §4.3 · p. 7, Figure 6 · p. 9
  • Models trained with 4 players generalize to 6-, 8-, and 10-player inference without retraining or additional parameters, with stronger consistency, controllability, and visual quality than Solaris and γ-World.Multics is also reported to maintain robust multiplayer generation on EgoTeam.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4 ABLATION STUDY

  • Hierarchical communication improves consistency and action/camera controllability; centralized communication is more susceptible to heterogeneous information mixing, while group-based communication lacks intermediate group-level representations.The qualitative ablation reports group-based inconsistencies when revisiting scenes and across player views.Source: §4.4 · p. 9, Figure 5 · p. 9
  • Fixed grouping yields lower GeoKPM and reduced action and camera accuracy than dynamic grouping, while random grouping substantially decreases GeoKPM and camera accuracy.Dynamic grouping organizes relevant players for information exchange.Source: §4.4 · p. 9
  • Reducing the inference communication budget below 1.0× decreases GeoKPM, whereas increasing it beyond 1.0× leaves GeoKPM relatively stable.The model is trained with a 1.0× default budget, and inference budgets from 0.6× to 1.4× are tested without retraining.Source: §4.4 · pp. 9–10, Figure 7 · p. 9

5 CONCLUSION

Multics structures multi-player information exchange hierarchically and uses shared group-token parameters and camera-trajectory-based dynamic grouping. Experiments report robust generation and scaling to more players.

5 CONCLUSION

  • Multics structures information exchange across player, group, and global levels, using group tokens to aggregate locally relevant player information while retaining cross-group communication.The framework targets robust multi-player video generation.Source: §5 · p. 10
  • Shared group-token parameters accommodate varying player counts without new group-specific parameters, while budget-constrained dynamic grouping uses recent camera trajectories to select players sharing group representations.Source: §5 · p. 10
  • Experiments report state-of-the-art performance in robust multi-player video generation, with cross-player consistency, controllability, and interference robustness while scaling to more players.Source: §5 · p. 10

A IMPLEMENTATION DETAILS

The implementation uses a two-stage diffusion-forcing training schedule and camera-based, budget-constrained grouping to structure communication as player counts vary. Grouping uses recent camera cues to select relevant players while keeping communication within the centralized baseline budget.

A IMPLEMENTATION DETAILS

  • Training uses 10K steps with a 7-frame attention window, then 25K steps with a 21-frame window, at a learning rate of 3 × 10−5.The model is trained on 16 GPUs with an effective batch size of 16 multi-player samples; inference uses 165-frame videos and 50 denoising steps.Source: A IMPLEMENTATION DETAILS · p. 15

B DETAILS OF DYNAMIC GROUPING

  • Pairwise relevance combines camera proximity and viewing direction with mutual visibility, using recent causal camera history and smoothing.The relevance matrix selects communication topology; self-attention learns the information exchanged between connected representations.Source: B DETAILS OF DYNAMIC GROUPING · p. 15
  • Candidate groups favor high-relevance player relations and player coverage, subject to group-capacity and communication-budget constraints.Groups may overlap under limits: each player belongs to at most two groups, and two groups share at most one player; uncovered players communicate through global tokens.Source: B DETAILS OF DYNAMIC GROUPING · p. 16
  • 4-to-6-player evaluation shows the smallest degradation on six of eight metrics, with lower degradation indicating better performance preservation.Negative degradation values indicate improvement.Source: Figure 8 · p. 17
  • The time-varying membership mask connects players to assigned local hubs and gives direct attention to players sharing a group.The complete clip-level mask is checked against the centralized communication cost; memberships can change at grouping updates.Source: B DETAILS OF DYNAMIC GROUPING · pp. 16–17

C.1 ADDITIONAL RESULTS ON GENERALIZATION TO MORE PLAYER COUNTS

Multics preserves performance better than matched 6-player-trained models when extrapolating from 4-player training to 6-player inference. It has the smallest degradation on most reported metrics.

C.1 ADDITIONAL RESULTS ON GENERALIZATION TO MORE PLAYER COUNTS

  • Multics has the smallest performance degradation on most metrics when models trained with 4 players are evaluated with 6 players, compared with matched 6-player training.Lower relative degradation indicates better preservation of performance.Source: §C.1 · p. 17

C.2 ADDITIONAL ABLATION STUDY

The shared group-token design outperforms non-shared alternatives in a 4-player-training to 6-player-evaluation ablation. Additional qualitative results are available in a compressed supplementary video.

C.2 ADDITIONAL ABLATION STUDY

  • Shared group-token parameters consistently outperform non-shared variants across all metrics when evaluating 6-player scenes after 4-player training.The non-shared model’s unseen third-group tokens were initialized by copying either existing group or averaging both; sharing makes group slots exchangeable.Source: §C.2 · p. 17

C.3 MORE QUALITATIVE RESULTS

  • The supplementary video provides additional qualitative results and is compressed to 720p because of upload-size limits.Source: §C.3 · p. 17

C.4 QUALITATIVE RESULTS ON CROSS-DOMAIN APPLICABILITY

On EgoTeam, Multics is reported to improve cross-view consistency and robustness over γ-World in multi-robot egocentric video generation. The results extend the evaluation beyond gaming environments to reconstructed indoor scenes.

C.4 QUALITATIVE RESULTS ON CROSS-DOMAIN APPLICABILITY

  • On EgoTeam, Multics better preserves cross-view consistency and robustness across multiple agents than γ-World.The qualitative comparison covers multi-robot egocentric video generation.Source: §C.4 · p. 18, Figure 9 · p. 18
  • EgoTeam comprises multi-robot egocentric videos rendered in reconstructed real-world indoor scenes at 256×256 resolution and 5 FPS.Source: §C.4 · p. 18
  • The paper reports that Multics outperforms the baselines across multiple metrics on EgoTeam.Source: §C.4 · p. 18

D LIMITATIONS

Multics focuses on cross-player consistency, controllability, and robustness to cross-event interference; persistent world states across players over long horizons remain challenging, and the pretrained video foundation model can produce artifacts under large, dynamic human motions.

D LIMITATIONS

  • Persistent world states across multiple players over long horizons remain a challenging direction beyond the paper’s focus on consistency, controllability, and interference robustness.Source: D LIMITATIONS · p. 18
  • Multics inherits pretrained video foundation-model limitations, including visual artifacts under large, dynamic human motions.Source: D LIMITATIONS · p. 18

Related papers

Loading 2610.17201v1…