Source-linked AI summary
InfinityStory: Unlimited Video Generation with World Consistency and Character-Aware Shot Transitions
Mohamed Elmoghany, Liangbing Zhao, Xiaoqian Shen, Subhojyoti Mukherjee, Yang Zhou, Gang Wu, Viet Dac Lai, Seunghyun Yoon, Ryan Rossi, Abdullah Rashwan, Puneet Mathur, Varun Manjunatha, Daksh Dangi, Chien Nguyen, Nedim Lipka, Trung Bui, Krishna Kumar Singh, Ruiyi Zhang, Xiaolei Huang, Jaemin Cho, Yu Wang, Namyong Park, Zhengzhong Tu, Hongjie Chen, Hoda Eldardiry, Nesreen Ahmed, Thien Nguyen, Dinesh Manocha, Mohamed Elhoseiny, Franck Dernoncourt
TL;DR
Long-form storytelling video generation struggles with stable backgrounds, smooth multi-subject shot transitions, and hour-long scalability. InfinityStory combines location-grounded background generation, hierarchical planning, and a transition model trained on 10,000 multi-subject videos. It ranks first overall on VBench with an average rank of 2.80 and the highest Subject Consistency and Background Consistency scores.
Problem
Long-form video synthesis lacks reliable background consistency, multi-subject shot transitions, and scalability to hour-long narratives.
Method
InfinityStory binds scenes to fixed locations, injects persistent backgrounds into character-conditioned keyframes, and trains a first–last-frame-to-video transition model on 10,000 multi-subject sequences.
Results
2.80 average rank, 82.11 Subject Consistency, and 88.94 Background Consistency are InfinityStory’s strongest reported VBench results among baselines.
Takeaways & Limitations
Explicit location binding and supervised multi-subject transitions provide a foundation for scalable cinematic storytelling with stable environments and coherent character dynamics.
Takeaways & Limitations
Image Quality and Aesthetic scores are slightly lower than some baselines because the pipeline uses 480p resolution and image editing introduces visual artifacts.
Abstract
from arXiv · showhide
Generating long-form storytelling videos with consistent visual narratives remains a significant challenge in video synthesis. We present a novel framework, dataset, and a model that address three critical limitations: background consistency across shots, seamless multi-subject shot-to-shot transitions, and scalability to hour-long narratives. Our approach introduces a background-consistent generation pipeline that maintains visual coherence across scenes while preserving character identity and spatial relationships. We further propose a transition-aware video synthesis module that generates smooth shot transitions for complex scenarios involving multiple subjects entering or exiting frames, going beyond the single-subject limitations of prior work. To support this, we contribute with a synthetic dataset of 10,000 multi-subject transition sequences covering underrepresented dynamic scene compositions. On VBench, InfinityStory achieves the highest Background Consistency (88.94), highest Subject Consistency (82.11), and the best overall average rank (2.80), showing improved stability, smoother transitions, and better temporal coherence.
1. Introduction
InfinityStory targets persistent background consistency, smooth multi-subject transitions, and scalable long-form storytelling video generation. Its background-aware pipeline and transition model improve consistency and transition smoothness, with strong VBench performance.
- Research gap: Current agentic storytelling systems improve narrative coordination but typically generate only 4–16-second clips without long-term visual consistency.They also lack explicit background preservation and multi-subject transition modeling, causing scene drift and jarring clip boundaries.
- Proposed framework: InfinityStory assigns each scene to a fixed location and fuses characters with the corresponding background to preserve scene identity across shots.This background-consistent generation strategy is designed for extended narrative sequences.
- Proposed framework: A 10,000-video multi-subject transition dataset supports a first–last-frame-to-video model for smooth entry, exit, and replacement events between shots.The dataset covers transition scenarios that prior methods do not systematically model.
- Results: 88.94 Background Consistency and 82.11 Subject Consistency are the highest reported values among baselines, while the overall VBench average rank is 2.80.The framework also improves stability and transition smoothness on automatic metrics and human studies.
- Proposed framework: InfinityStory scales narrative video generation toward hour-long sequences while preserving story structure and cinematographic coherence across hundreds of shots.The framework combines hierarchical planning, background preservation, and supervised transition modeling.
2. Related works
Prior storytelling systems use hierarchical planning and cross-shot conditioning, but InfinityStory emphasizes location-grounded backgrounds and multi-subject transition modeling. Its dataset-generation process combines agentic prompting, VLM filtering, and transition-video supervision.
- Agentic storytelling systems: Recent systems coordinate script writing, shot design, animation, clip selection, and cinematography through hierarchical or multi-agent planning.Examples include MovieAgent, MAViS, AniMaker, MovieDreamer, Captain Cinema, and FilMaster.
- Image-based story visualization: StoryGPT-V generates coherent image sequences, but its outputs remain discrete images and its training and application are limited to a single-movie domain.This leaves continuous long-form video generation and broader domain generalization unresolved.
- Video consistency and transitions: Existing video methods use long-context attention, interleaved conditioning, or mask-based diffusion, yet stable backgrounds can still shift after a few shots.These approaches address cross-shot coherence or transitions without fully solving persistent scene identity.
- InfinityStory: InfinityStory generates fixed locations, binds scenes to them, and fuses selected backgrounds with characters to construct consistent keyframes.The keyframes anchor subsequent video generation across shots sharing a scene.
- InfinityStory: InfinityStory addresses abrupt character changes by modeling multi-subject entry, exit, and replacement events with a dedicated transition dataset and model.Its transition formulation disallows sudden appearance or disappearance except when moving between scenes.
3. Methods
InfinityStory combines hierarchical planning, location-grounded background injection, cross-shot memory, and transition-aware video generation to support coherent long-form narratives. Its transition module targets multi-subject entry, exit, replacement, and repositioning scenarios through synthetic training data and explicit transition logic.
- Multi-Agent Narrative: The framework decomposes stories into chapters, locations, scenes, and shots, propagating global constraints through a hierarchical multi-agent pipeline.Each scene binds to a fixed location, while shots receive normalized directives and transition metadata.
- Sequential Generation: Cross-shot memory accumulates identity and layout information, while odd shots use I2V and even shots use FLF2V to bridge adjacent narrative shots.Transition shots use endpoint keyframes composed with the same background to reduce scene drift.
- Background Consistency: Canonical location backgrounds are fused with shot-specific character references to create keyframes that anchor subsequent video generation.The pipeline uses T2I backgrounds, I2I compositing, and I2V generation while penalizing perceptual background drift across frames.
- Multi-Character Transitions: CMTS addresses FLF2V failures on character entry, exit, and replacement by conditioning transitions on explicit movement logic.Transition metadata distinguish entry, exit, no change, combination, and replacement scenarios.
- Multi-Character Transitions: 10,000 synthetic transition videos are generated, filtered with a vision-language model, and used to finetune a transition model on underrepresented multi-character cases.A balanced prior ensures coverage of rare transition scenarios.
4. Experiments
InfinityStory performs strongly on VBench consistency and overall ranking, while image quality and aesthetics trail some baselines because of resolution and editing artifacts.
- 2.80 average rank: InfinityStory achieves the best overall VBench ranking among all baselines.
- 82.11 Subject Consistency and 88.94 Background Consistency: InfinityStory records the highest scores among comparison baselines.
- Image Quality and Aesthetic scores are slightly lower than some baselines because the pipeline operates at 480p and editing introduces degradation.
- Motion smoothness remains competitive, although VBench may not fully capture higher-level narrative quality or story-driven temporal logic.
- Removing background injection reduces all metrics, particularly Subject Consistency by 3.6 and Background Consistency by 1.6.
5. Conclusions
InfinityStory combines hierarchical planning, location-grounded background injection, and supervised multi-subject transitions for long-form narrative video generation. It achieves strong consistency and ranking results, but its transition model remains limited on unseen character combinations and complex storylines.
- InfinityStory enables long-form narrative video generation with world-level consistency and smooth multi-character shot transitions.
- The framework combines hierarchical multi-agent planning, location-grounded background injection, and a transition model trained on a curated multi-subject dataset.
- 88.94 Background Consistency, 82.11 Subject Consistency, and 2.80 VBench average rank are the paper’s highest reported results against baselines.
- The FLF2V transition model has limited generalization to unseen character combinations and complex storylines.
S1. Experiments
Human evaluations favor InfinityStory across scene coherence, transitions, character consistency, motion smoothness, and visual quality. Its moderate motion level reflects a design emphasis on continuity, while implementation choices support natural transitions and reliable character identity.
- InfinityStory outperforms MovieAgent and Video-Gen-of-Thought across every evaluated human-study metric.
- 59.32 background consistency, 55.93 motion smoothness, and 50.85 image quality and aesthetic lead the human-study comparisons.
- 45.76 smooth transitions and 42.37 character consistency exceed the corresponding scores of both comparison systems.
- 53.35 Dynamic Degree is moderate, reflecting emphasis on scene continuity and multi-shot consistency rather than maximizing motion amplitude.
- Generating transitions from the preceding I2V output lets FLF2V begin from an actual visual state instead of a predefined keyframe.
- Replacing Qwen Image Edit 2509 with OmniGen2 preserved character identity more reliably during multi-image injection.
S2. Methods
The methods combine agentic story and shot planning with transition-focused data generation, filtering, and human evaluation. Planning structures narrative and shot continuity, while the dataset and evaluation target multi-subject transitions and video quality.
- Agentic Story Planning: Agentic story planning converts scripts into structured chapters and supports consistent narrative structure and character tracking.
- Human Evaluation: Human evaluations compare three methods across background consistency, transition smoothness, character identity, motion naturalness, and overall visual quality.
- Shot-Level Planning: Shot-level planning specifies emotions, poses, interactions, camera movement, cinematography, dialogue timing, and transition logic for adjacent shots.
- Multi-Subject Transitions: The pipeline uses structured intermediate representations to reason about character entry, exit, and repositioning while preserving scene continuity.
- Transition Dataset: 10,000 multi-subject transition sequences support training, with failed samples filtered automatically and successful transitions used to train the FLF2V module.