Source-linked AI summary
FlowAct-R1: Towards Interactive Humanoid Video Generation
Lizhen Wang, Yongming Zhu, Zhipeng Ge, Youwei Zheng, Longhao Zhang, Tianshu Hu, Shiyang Qin, Mingshuang Luo, Jiaxu Zhang, Xin Chen, Yulong Wang, Zerong Zheng, Jianwen Jiang, Chao Liang, Weifeng Chen, Xing Wang, Yuan Zhang, Mingyuan Gao
TL;DR
Interactive humanoid video generation seeks lifelike agents that remain responsive while producing high-fidelity, temporally consistent behavior. FlowAct-R1 combines MMDiT-based streaming, chunkwise diffusion forcing, structured memory, distillation, and system optimization, achieving 25fps 480p generation with approximately 1.5s TTFF and strong behavioral vividness. The paper also notes risks of deceptive or harmful misuse requiring controlled deployment.
Problem
Interactive humanoid video generation must combine low-latency streaming with long-duration temporal consistency and natural transitions across behavioral states.
Method
FlowAct-R1 adapts an MMDiT backbone with chunkwise diffusion forcing, structured memory, multimodal control, distillation, and system-level inference optimizations.
Results
25fps at 480p with approximately 1.5s TTFF accompanies reported long-duration streaming, superior behavioral naturalness, and high-quality synthesis across diverse character styles.
Takeaways & Limitations
FlowAct-R1 supports continuous interactive video with fine-grained behavioral control, natural state transitions, and generalization across character styles from a single reference image.
Takeaways & Limitations
The method could be misused to fabricate deceptive or harmful content, so the authors propose access control limited to verified entities.
Abstract
from arXiv · showhide
Interactive humanoid video generation aims to synthesize lifelike visual agents that can engage with humans through continuous and responsive video. Despite recent advances in video synthesis, existing methods often grapple with the trade-off between high-fidelity synthesis and real-time interaction requirements. In this paper, we propose FlowAct-R1, a framework specifically designed for real-time interactive humanoid video generation. Built upon a MMDiT architecture, FlowAct-R1 enables the streaming synthesis of video with arbitrary durations while maintaining low-latency responsiveness. We introduce a chunkwise diffusion forcing strategy, complemented by a novel self-forcing variant, to alleviate error accumulation and ensure long-term temporal consistency during continuous interaction. By leveraging efficient distillation and system-level optimizations, our framework achieves a stable 25fps at 480p resolution with a time-to-first-frame (TTFF) of only around 1.5 seconds. The proposed method provides holistic and fine-grained full-body control, enabling the agent to transition naturally between diverse behavioral states in interactive scenarios. Experimental results demonstrate that FlowAct-R1 achieves exceptional behavioral vividness and perceptual realism, while maintaining robust generalization across diverse character styles.
1 Introduction
Interactive humanoid video generation requires low-latency streaming, long-form temporal consistency, and natural transitions across behavioral states. FlowAct-R1 is introduced to address these requirements while combining real-time performance with high-fidelity behavioral expressivity.
- Interactive humanoid video generation must respond to conversational audio and text while maintaining naturalistic visual behavior.
- Long-duration interaction makes streaming generation, low latency, visual quality, and temporal consistency difficult to maintain simultaneously.
- Existing lip-synchronization methods align mouth movements with speech but lack fine-grained full-body gesture control.
- FlowAct-R1 combines streaming, real-time performance, and high-fidelity behavioral expressivity in interactive humanoid video generation.
- Chunkwise diffusion forcing and self-forcing address error accumulation and promote long-term temporal consistency during streaming.
- 25fps at 480p with approximately 1.5s TTFF is achieved through efficient distillation, operator optimizations, and parallel computing.
- Holistic audio and text control supports natural transitions between diverse interactive states and behavioral vividness across character styles.
2 Approach
FlowAct-R1 adapts an MMDiT backbone for indefinite-length streaming through chunkwise generation, structured memory, multimodal conditioning, and staged acceleration. Its pipeline combines training strategies and system optimizations to support temporally coherent, low-latency video synthesis.
- FlowAct-R1 uses Seedance, an MMDiT backbone with native crossmodal alignment, for streaming humanoid video generation.
- Audio and text are fused with visual representations through cross-attention to control lip synchronization, body dynamics, and detailed behaviors.
- A structured stream buffer stores a reference latent, long-term memory queue, short-term memory latent, and denoising stream.
- 0.5 seconds of video are continuously produced per 0.5 seconds of wall-clock time using a 3-NFE inference cycle.
- The three-stage curriculum performs autoregressive adaptation, joint audio-motion training, and distillation to make the model streaming-compatible.
- Cross-prompt training models transitions between differing text prompts, while weighted loss preserves coherent first-segment image-to-video initialization.
- Fake-causal attention keeps reference and memory information stable while allowing the denoising stream to access those components.
3 Experiments
FlowAct-R1 is evaluated against KlingAvatar 2.0, LiveAvatar, and Omnihuman-1.5 in a 20-participant GSB user study. The evaluation covers naturalness, lip synchronization, frame stability, and motion richness, with the method reported to outperform competitors across streaming and behavioral criteria.
- FlowAct-R1 is compared with KlingAvatar 2.0, LiveAvatar, and Omnihuman-1.5, which differ in duration, streaming support, and motion-repetition issues.
- The user study uses the GSB metric to evaluate motion naturalness, lip-sync accuracy, frame structure stability, and motion richness.
- 20 participants took part in the user study, with audio durations matched to each method’s maximum supported video duration.
- 25fps at 480p and approximately 1.5s TTFF accompany FlowAct-R1’s reported advantages in long-duration streaming and behavioral naturalness.
- Figure 3 reports the percentage of user votes favoring FlowAct-R1 over each competing method using orange GSB segments.
4 Conclusion
FlowAct-R1 supports continuous, responsive humanoid video generation with temporal consistency and visual fidelity. The framework also reports behavioral vividness, perceptual realism, and diverse character-style synthesis, alongside responsible-deployment safeguards.
- FlowAct-R1 enables streaming, arbitrary-duration humanoid video generation while maintaining temporal consistency and visual fidelity.
- The model delivers behavioral vividness and perceptual realism, including natural transitions across complex interactive states.
- FlowAct-R1 maintains high-quality synthesis across diverse character styles from a single reference image.
- The authors identify potential misuse for deceptive or harmful content and commit to access controls restricting core models to verified entities.