PaperlayerView exact revision on arXiv

Source-linked AI summary

STORYANCHORS: Generating Consistent Multi-Scene Story Frames for Long-Form Narratives

Bo Wang, Haoyang Huang, Zhiying Lu, Fengyuan Liu, Guoqing Ma, Jianlong Yuan, Yuan Zhang, Nan Duan, Daxin Jiang

arXiv:2505.08350v2cs.CVcs.AI

TL;DR

Long-form story generation remains challenged by limited global coherence and consistency across evolving scenes. STORYANCHORS combines bidirectional story-frame prediction, multi-event labeling, and progressive training; it reports stronger results than open-source models across several dimensions and comparable narrative consistency and richness to GPT-4o.

  • Problem

    Existing methods struggle with complex story-driven videos, long-range coherence, and consistent multi-scene frame generation.

    Source: §1 · pp. 1–2

  • Method

    STORYANCHORS predicts story frames using bidirectional context and combines multi-event labeling with progressive training to capture global narrative structure and event-level dynamics.

    Source: §1 · pp. 2–3, §3.2 · p. 5, §3.3 · p. 6

  • Results

    STORYANCHORS outperforms open-source models across narrative, consistency, alignment, and diversity, with narrative consistency and story richness comparable to GPT-4o.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    By decoupling keyframe prediction from intermediate-frame synthesis, STORYANCHORS offers a basis for scalable video creation with strong temporal consistency.

    Source: §5 · p. 9

    Its reliance on TI2V-based generation limits adaptation to complex scenes and can result in weak transitions and inconsistent narrative continuity.

    Source: §5 · p. 9

Abstract

from arXiv · show

This paper introduces StoryAnchors, a unified framework for generating high-quality, multi-scene story frames with strong temporal consistency. The framework employs a bidirectional story generator that integrates both past and future contexts to ensure temporal consistency, character continuity, and smooth scene transitions throughout the narrative. Specific conditions are introduced to distinguish story frame generation from standard video synthesis, facilitating greater scene diversity and enhancing narrative richness. To further improve generation quality, StoryAnchors integrates Multi-Event Story Frame Labeling and Progressive Story Frame Training, enabling the model to capture both overarching narrative flow and event-level dynamics. This approach supports the creation of editable and expandable story frames, allowing for manual modifications and the generation of longer, more complex sequences. Extensive experiments show that StoryAnchors outperforms existing open-source models in key areas such as consistency, narrative coherence, and scene diversity. Its performance in narrative consistency and story richness is also on par with GPT-4o. Ultimately, StoryAnchors pushes the boundaries of story-driven frame generation, offering a scalable, flexible, and highly editable foundation for future research.

1 Introduction

STORYANCHORS addresses the difficulty of generating multi-scene narratives with consistent subjects and coherent long-range progression. It combines bidirectional story-frame prediction with multi-event labeling and progressive training, and reports stronger open-source-model comparisons across several dimensions.

  • Autoregressive methods improve adjacent-frame consistency but can produce repetitive scenes and struggle with error accumulation, global planning, and long-range coherence.Source: §1 · p. 1
  • Existing story-frame methods face limited frame counts or repetitive character-centric views, while consistency, semantic alignment, and narrative coherence become harder as sequences grow.Source: §1 · pp. 1–2
  • STORYANCHORS uses bidirectional prediction and multi-event labeling with progressive training to capture narrative events across evolving scenes while maintaining temporal coherence.Source: §1 · pp. 2–3
  • STORYANCHORS generates multiple frames in one pass and outperforms open-source models across narrative, consistency, alignment, and diversity; narrative consistency and story richness are comparable to GPT-4o.Source: §1 · pp. 2–3
  • Its frames are editable and expandable, supporting longer sequences and manual narrative modifications.Source: §1 · p. 3

2 Related Work

Prior video-generation approaches can create high-quality frames or extend video length, but the cited methods have limits in global planning, scene diversity, and narrative depth. STORYANCHORS instead treats semantically rich, globally consistent story frames as structural anchors for multi-scene video.

  • MLLM-based frame generation often relies on past visual context, limiting global planning and making scene diversity, temporal coherence, and narrative structure difficult.Source: §2 · p. 3
  • Scene-level video generation includes keyframe-based and autoregressive methods, but keyframe approaches can lack global control and coherent transitions across shots.Source: §2 · p. 3
  • Autoregressive methods extend video length but still lack explicit global planning of key visual anchors and struggle to maintain coherent narratives across scenes.Source: §2 · p. 3
  • STORYANCHORS uses globally consistent, semantically rich story frames as structural anchors for coherent multi-scene, minute-scale videos.Source: §2 · p. 3

3 Approach

STORYANCHORS combines bidirectional story-frame prediction, multi-event labeling, and progressive training to generate coherent, story-driven videos with multi-scene continuity and flexible transitions.

3 Approach

  • The approach combines Bidirectional Story Frame Prediction, Multi-Event Story Frame Labeling, and Progressive Story Frame Training to support coherent, story-driven videos.Source: §3 · pp. 3–4

3.1 Bidirectional Story Frame Prediction

  • The prediction network adds condition frames or zero padding and a mask to the latent input; condition embeddings control generation, and image-to-video interpolation synthesizes intermediate frames.Source: §3.1 · p. 4

3.2 Multi-Event Story Frame Labeling

  • Multi-event labeling splits sequences into subsets of up to 12 frames, annotates events and frame ranges, then combines event descriptions and representative frames into coherent captions with consistency, subject, and style fields.Source: §3.2 · p. 5

3.3 Progressive Story Frame Training

  • Progressive training moves from global-caption pretraining to event-level fine-tuning, then trains future story-frame prediction from initial frames and prompts for story expansion.Source: §3.3 · pp. 5–6

4 Experiments

The experiments compare STORYANCHORS with open-source methods and GPT-4o, test continuation under varied frame conditions, and examine multi-event prompts. Results report strengths in subject consistency and scene diversity, while multi-event descriptions preserve event completeness and order in the examples.

4.1 Implement Details

  • The experiments use a 42-story multi-event benchmark generated with Claude and refined manually, with Step-Video-T2V-4B as the base model.Source: §4.1 · p. 6

4.2 Comparisons of Consistent Story Frames Generation

  • When generating 10 frames, STORYANCHORS excels in subject consistency and scene diversity; against GPT-4o, it scores slightly lower in NQ and PFA but achieves comparable overall results.Figure 5 summarizes open-source methods across five Gemini-evaluated metrics; Table 1 reports win/tie/loss comparisons based on GPT-4o-generated story frames.Source: §4.2 · pp. 7–8, Figure 5 · p. 7, Table 1 · p. 7

4.3 Story Frames Editing and Temporal Extension

  • At inference, the model continues coherently from multiple frames despite training only on first-frame inputs, and preserves subject identity across different single-frame conditions.Source: §4.3 · p. 8

4.4 Effectiveness of Multi-Event Prompt

  • Compared with global prompts, multi-event descriptions capture significant story moments in the correct sequence in both examples.Source: §4.4 · p. 9

5 Conclusion

STORYANCHORS predicts globally coherent story frames using bidirectional context and separates keyframe prediction from intermediate-frame synthesis. The conclusion notes a TI2V-related limitation for complex scenes and tasks.

  • STORYANCHORS predicts globally coherent story frames for consistent, multi-scene videos.Source: §5 · p. 9
  • Separating keyframe prediction from intermediate-frame synthesis is presented as a scalable approach to video creation with temporal consistency.Source: §5 · p. 9
  • The conclusion identifies reliance on TI2V-based generation as limiting adaptation to complex scenes or tasks, with weak transitions and inconsistent narrative continuity.Source: §5 · p. 9

A Evaluation Metrics

The evaluation uses five criteria to assess story quality, subject continuity, prompt alignment, and shot and scene variety. Ratings are assigned through multi-round comparisons with order effects mitigated by reversing presentation order.

  • Narrative Quality assesses whether frames convey a clear, coherent, complete story arc, while Subject Consistency measures continuity of key subject attributes.Source: A Evaluation Metrics · p. 12
  • Prompt-Frame Alignment measures how accurately frames reflect prompt content, including scene, character, and emotional setting.Source: A Evaluation Metrics · p. 12
  • Shot Diversity evaluates camera-angle and composition variation, while Scene Diversity measures the variety and progression of environments across frames.Source: A Evaluation Metrics · p. 12
  • The multi-round evaluation assigns ratings on a 1–5 scale and reverses model order to mitigate order effects.Source: A Evaluation Metrics · p. 12

B Evaluation Criteria

The evaluation criteria describe five-point ranges for judging narrative continuity, subject consistency, prompt alignment, shot variety, and scene diversity. Higher ranges correspond to stronger continuity or richer, more appropriate visual variation.

  • Narrative ratings range from no discernible plot or continuity to coherent key moments, emotional changes, and film-like narrative rhythm.Source: B Evaluation Criteria · p. 12
  • Subject-consistency ratings range from unrecognizable changes in appearance to uniform, natural details and changes across keyframes.Source: B Evaluation Criteria · p. 13
  • Prompt-alignment ratings range from ignoring core prompt elements to accurately reflecting explicit descriptions and implied atmosphere, style, and emotions.Source: B Evaluation Criteria · p. 13
  • Shot-diversity ratings range from repetitive angles and compositions to varied shot language serving narrative and emotional expression.Source: B Evaluation Criteria · p. 13
  • Scene-diversity ratings range from a single repetitive setting to rich environments with transitions suited to the narrative.Source: B Evaluation Criteria · p. 13

Related papers

Loading 2505.08350v2…