Source-linked AI summary
Wan-Streamer v0.2: Higher Resolution, Same Latency
Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, Yuzheng Wang, Zoubin Bi
TL;DR
Wan-Streamer v0.2 addresses the limited visual range of low-resolution interactive streams for scene-grounded audio-visual conversation. It upgrades the stream to 640×368 using a context-parallel performer while retaining approximately 200 ms model-side latency and approximately 550 ms remote interaction latency with networking.
Problem
The preliminary 192p stream limits visual range, compressing body posture, nearby objects, and scene context needed for scene-grounded interaction.
Method
The system keeps a single-GPU thinker and uses a Ulysses-style multi-GPU performer with pre-sharded K/V caches and parallel high-resolution video denoising.
Results
640×368 output is achieved at approximately 200 ms model-side latency and approximately 550 ms remote interaction latency with a 350 ms network budget.
Takeaways & Limitations
The higher-resolution stream supports clearer video-call interaction and scene-grounded mid-shot agents with legible posture, gaze, hands, objects, and local scene layout.
Takeaways & Limitations
Runtime comparisons use the v0.1 response-boundary convention because public real-time systems report different latency endpoints.
Abstract
from arXiv · showhide
We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.
1 Introduction
Wan-Streamer v0.2 preserves the native-streaming formulation while increasing output resolution from 192×336 to 640×368 at 25 FPS and maintaining approximately 200 ms model-side response latency. Its revised serving topology supports scene-grounded mid-shot agents with more legible body and surrounding-scene context.
- Formulation: The model retains the v0.1 native-streaming formulation, representing language, audio, and video on a shared causal timeline coordinated by block-causal attention.The upgrade changes output resolution and deployment strategy without changing the underlying interaction formulation.
- Resolution and latency: 192×336 to 640×368 at 25 FPS, while maintaining approximately 200 ms model-side response latency.The upgrade preserves the streaming interaction cadence while increasing the interactive output resolution.
- Visual interaction scope: v0.2 supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation.The visual scope expands beyond portrait-like calls toward situated conversations grounded in surrounding context.
- Serving topology: The serving topology keeps a single-GPU thinker and moves high-resolution latent video generation into a Ulysses-style context-parallel multi-GPU performer.The thinker handles streaming perception, language/state updates, K/V-cache construction, and final causal decoding, while the performer handles the expensive video path.
2 Upgrade Design
Wan-Streamer v0.2 preserves the core end-to-end causal streaming formulation and latency budget while increasing visual output from 192p to 640×368. The upgrade targets wider, scene-grounded conversational settings that retain identity, gaze, posture, nearby objects, and scene layout during real-time interaction.
- Core formulation: The core Wan-Streamer formulation remains an end-to-end causal stream in which observations and generated responses update a shared history.User text, audio, and video observations condition agent text, speech, and video responses; generated audio-video latents re-enter history after each unit.
- Visual upgrade: 640×368 replaces 192p as the visual target, expanding the model from mostly close-up calls to wider, scene-grounded conversational settings.The higher-resolution mid-shot setting requires preserving identity, gaze, hand and torso posture, local objects, and scene layout while listening and speaking in real time.
- Version-level changes: The end-to-end streaming formulation and latency budget stay fixed, while output resolution, visual format, and serving topology change.This version-level summary identifies the preserved and changed components of the v0.2 upgrade.
3 Latency-Preserving Serving
Wan-Streamer v0.2 preserves low-latency serving by keeping perception, language/state updates, K/V construction, and decoding on one thinker GPU while distributing expensive high-resolution latent video generation across a Ulysses-style performer group. The schedule keeps real-time units within 160 ms and model-side response latency at approximately 200 ms.
- Thinker–performer split: The thinker remains on one GPU for causal encoding, language/state updates, K/V-cache construction, and final audio-video decoding.The language/state computation conditions generation through the returned K/V cache.
- Thinker–performer split: The performer uses Ulysses-style context parallelism for expensive 640×368 latent generation, splitting the long video sequence across ranks and retaining pre-sharded K/V caches.Ulysses all-to-all/gather communication stays within the performer group around attention.
- Streaming schedule: At each streaming unit, the thinker sends a new K/V slice while receiving prior latents for decoding, and performer ranks denoise the next unit using local full-history cache shards.The high-resolution latent video is the main sequence-parallel path.
- Latency constraints: 160 ms is the real-time unit budget for performer computation, thinker–performer transfers, and intra-performer Ulysses communication.The schedule separates throughput from response latency by fitting these operations within one streaming unit.
- Latency constraints: Approximately 200 ms is the model-side signal-to-signal response latency through encoding, state update, latent generation, and decoding.Additional v0.2 work is concentrated in the context-parallel performer rather than slowing the interactive loop.
4 Experiments
Experiments show that Wan-Streamer v0.2 preserves approximately 200 ms model-side latency while producing 640×368 video at 25 FPS under the v0.1 response boundary. Qualitative inspection finds clearer close-up calls and scene-grounded mid-shot agents in the same low-latency streaming setting.
- Latency and runtime protocol: Approximately 200 ms model-side latency is maintained while producing 640×368 video at 25 FPS.Latency is measured from availability of a 160 ms user streaming unit to decoding the corresponding audio-video response unit for emission.
- Latency and runtime protocol: The experiment retains the v0.1 measurement convention because public systems report differing latency endpoints.The reported runtime uses the same response boundary as Wan-Streamer v0.1.
- Qualitative visual observations: Qualitative inspection of 640×368 conversations focuses on stability and legibility across listening and speaking intervals.Inspected aspects include facial detail, gaze, mouth motion, hands, posture, nearby objects, and local scene layout.
- Qualitative visual observations: The upgraded output format provides clearer close-up calls and scene-grounded mid-shot agents under the same low-latency streaming setting.These observations characterize the v0.2 output format.
5 Conclusion
Wan-Streamer v0.2 preserves v0.1’s native full-duplex formulation while increasing the interactive stream from 192×336 to 640×368 at approximately 200 ms model-side latency. A single-GPU thinker and Ulysses-style context-parallel performer absorb the added visual generation cost while maintaining low-latency streaming.
- Resolution and latency: 640×368 output, up from 192×336, is delivered at approximately 200 ms model-side latency.This preserves the native full-duplex formulation of v0.1.
- System design: The single-GPU thinker preserves the latency-critical loop, while the Ulysses-style performer handles high-resolution visual latent generation.Pre-sharded K/V caches and sequence parallelism support the performer’s visual latent-video denoising path.
- Interaction quality: The upgraded stream yields clearer video-call interaction and scene-grounded mid-shot agents under the same low-latency streaming setting.
Appendix · A Contributions and Acknowledgements · A.1 Core Contributors
The section identifies the paper’s core contributors by name. It lists nine individuals in the A.1 Core Contributors subsection.
- A.1 Core Contributors: The core contributors include Lianghua Huang and Zhi-Fan Wu.
- A.1 Core Contributors: The list also includes Yupeng Shi and Wei Wang.
- A.1 Core Contributors: Mengyang Feng and Junjie He are named as core contributors.
- A.1 Core Contributors: Chen-Wei Xie is identified among the core contributors.
- A.1 Core Contributors: Yu Liu is included in the core-contributor list.
- A.1 Core Contributors: Jingren Zhou completes the list of nine named core contributors.
A.2 Contributors
The paper lists 17 contributors alphabetically by first name.
- A.2 Contributors: 17 contributors are listed alphabetically by first name, including Ang Wang, Baole Ai, Chen Liang, and Zoubin Bi.The full list also includes Bang Zhang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, and Yuzheng Wang.