PaperlayerView exact revision on arXiv

Source-linked AI summary

A$^2$RD: Agentic Autoregressive Diffusion for Long Video Consistency

Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le

arXiv:2605.06924v1cs.CVcs.AI

TL;DR

Maintaining coherent, consistent narratives over long videos remains difficult because existing methods lose continuity across segments. A²RD uses a closed-loop agentic diffusion process with multimodal memory, adaptive generation, and self-improvement, achieving state-of-the-art consistency and narrative coherence across one- to ten-minute videos.

  • Problem

    Long video synthesis must preserve entities, environments, motion, and narratives over extended horizons, but existing approaches struggle with inter-segment continuity and visual-only memory.

  • Method

    A²RD synthesizes videos segment-by-segment through a Retrieve–Synthesize–Refine–Update loop combining multimodal memory, adaptive generation, and hierarchical self-improvement.

  • Results

    A²RD achieves state-of-the-art narrative coherence and visual consistency across public and LVBench-C benchmarks spanning 1-10 minute videos.

  • Takeaways & Limitations

    A²RD demonstrates a closed-loop approach for maintaining narrative coherence and visual consistency in long-horizon video synthesis.

  • Takeaways & Limitations

    A²RD requires component models with strong instruction-following and visual reasoning, while its verification rubrics may not generalize across creative styles, cultures, or domains.

Abstract

from arXiv · show

Synthesizing consistent and coherent long video remains a fundamental challenge. Existing methods suffer from semantic drift and narrative collapse over long horizons. We present A$^2$RD, an Agentic Auto-Regressive Diffusion architecture that decouples creative synthesis from consistency enforcement. A$^2$RD formulates long video synthesis as a closed-loop process that synthesizes and self-improves video segment-by-segment through a Retrieve--Synthesize--Refine--Update cycle. It comprises three core components: (i) Multimodal Video Memory that tracks video progression across modalities; (ii) Adaptive Segment Generation that switches among generation modes for natural progression and visual consistency; and (iii) Hierarchical Test-Time Self-Improvement that self-improves each segment at frame and video levels to prevent error propagation. We further introduce LVBench-C, a challenging benchmark with non-linear entity and environment transitions to stress-test long-horizon consistency. Across public and LVBench-C benchmarks spanning one- to ten-minute videos, A$^2$RD outperforms state-of-the-art baselines by up to 30% in consistency and 20% in narrative coherence. Human evaluations corroborate these gains while also highlighting notable improvements in motion and transition smoothness.

1. Introduction

Long video synthesis methods struggle with inter-segment consistency, continuity, and reliable tracking of entities and narratives. A2RD reframes synthesis as a training-free, closed-loop agentic process that combines multimodal memory, adaptive generation, and hierarchical self-improvement, alongside the LVbench-C benchmark for challenging long-horizon transitions.

  • Limitations of Existing Methods: Existing segment-based methods offer stronger narrative control but struggle to maintain inter-segment consistency and continuity.Visual-only conditioning in recent SAR methods is insufficient for reliably tracking entities and narratives across segments.
  • A2RD: A2RD reframes long video synthesis as a closed-loop, agentic process that synthesizes and self-improves videos autoregressively.The architecture is designed to enforce temporal consistency and narrative coherence over long horizons.
  • A2RD: A2RD integrates Multimodal Video Memory, Adaptive Segment Generation, and Hierarchical Self-Improvement to mitigate semantic drift and content collapse.These components form the architecture’s core pillars for maintaining consistency and coherence during ultra-long video synthesis.
  • LVbench-C: LVbench-C evaluates long-horizon video consistency under cyclical entity and environment appearances, including optional state changes.The benchmark addresses realistic narratives in which entities and environments appear, disappear, and reappear.
  • Evaluation: Extensive evaluations on LVbench-C and public benchmarks show that A2RD achieves state-of-the-art consistency and narrative coherence.The introduction reports that these results are obtained with self-improved generation, while experiments also analyze A2RD and its components.

2. Related Work

Prior long-video synthesis methods are passive and struggle with semantic drift, repetition, inter-segment consistency, and narrative progression. A2RD addresses these limitations by combining agentic memory, adaptive segment generation, and consistency-focused test-time self-improvement.

  • Long-Form Video Synthesis: Frame-based autoregressive methods preserve local visual fidelity but remain prone to semantic drift and content repetition.They condition frames or chunks on prior content through rolling KV caches, short-window attention with frame-level sinks, or initial-frame anchoring.
  • Test-Time Scaling for Generative Models: Existing video test-time scaling methods operate on single-segment quality and do not address inter-segment consistency or narrative progression.A2RD introduces efficient test-time algorithms targeting consistency and narrative coherence in multi-segment long-video synthesis.
  • Memory for LLM Agents: Memory is central to agentic systems because it maintains long-range dependencies across sequential decisions.LLM-agent memories store text, hidden representations, or graphs and typically incorporate retrieval mechanisms.
  • A2RD Architecture: A2RD retrieves relevant context, adaptively selects extrapolation or interpolation, synthesizes boundary frames and video segments, and applies hierarchical self-improvement.These operations form the per-segment architecture overview shown in Figure 4.

3. A2RD: Agentic Auto-Regressive Diffusion

A2RD is an agentic segment-based autoregressive architecture that synthesizes long videos through a Retrieve–Synthesize–Refine–Update closed loop. It combines multimodal memory, adaptive generation modes, hierarchical self-improvement, and memory-augmented prompt optimization to maintain coherence across segments.

  • Multimodal Video Memory: MVMem tracks evolving world states through textual states, reference frames, keyframes, and videos, enabling retrieval of relevant history and updates for future segments.Its textual states encode entity and environment evolution, spatial relations, and motion, while stored videos support transition and motion continuity.
  • Autoregressive Generation: A2RD synthesizes videos segment-by-segment using a Retrieve–Synthesize–Refine–Update loop, applying synthesis and refinement first to boundary frames and then to each video segment.After refinement, MVMem stores frames, videos, and textual states for subsequent generation.
  • Adaptive Segment Generation: A2RD adaptively selects extrapolation for spatio-temporally continuous contexts and interpolation otherwise, balancing natural progression against semantic drift and consistency failures.Interpolation is used for multi-shot contexts spanning different environments or when the next segment jumps to a new environment.
  • Hierarchical Self-Improvement: Hierarchical self-improvement extends refinement across boundary frames, video segments, and intra- and inter-segment coherence.Synthesized frames undergo frame-level refinement, while synthesized segments undergo video-level refinement before memory update.
  • Memory-Augmented Prompt Optimization: MAPO improves refinement by indexing successful and failed prompt refinements with prompts, refined prompts, rubric scores, and positive or negative labels.The prompt database entries are indexed by semantic embeddings of prompts and rubric scores.
  • Efficient Variant: A2RD-Par introduces parallel synthesis of boundary elements to improve efficiency.The supplied passage specifies parallel synthesis but does not provide a quantitative speedup.

4. Long Video Bench-Challenge (LVbench-C)

LVbench-C is introduced to stress-test long-video models on coherent world-state maintenance under non-linear entity and environment transitions. It provides structured 3-, 5-, and 10-minute scenarios built through human-in-the-loop generation and constrained refinement.

  • Benchmark Design: LVbench-C targets non-linear transitions in which entities and environments appear, disappear, and reappear, optionally with changed states across scenes.The benchmark is designed to test consistent and coherent world-state maintenance in these scenarios.
  • Benchmark Design: The benchmark contains 120 text-only scenarios spanning 3-, 5-, and 10-minute videos, with 50 samples each for 3- and 5-minute videos and 25 for 10-minute videos.Samples cover character, object, and environment scenarios, and scene descriptions either continue prior scenes or transition to new ones.
  • Dataset Construction: Each challenge type is generated with a professional screenwriter persona prompt, one human-designed demonstration, and a state-of-the-art MLLM.The construction pipeline is shared across 3-, 5-, and 10-minute scenarios.
  • Dataset Construction: Scenario constraints enforce meaningful story flow, predefined segment durations, cause-and-effect relationships, entity absence for at least n=10 segments, and realistic state changes.State changes are specified through visual markers such as positions, appearances, and conditions.

5. Experiments

Experiments evaluate A2RD on single-scene VBench-Long and the multi-scene LVbench-C benchmark using automatic and human metrics. A2RD improves long-horizon consistency, narrative coherence, and transition smoothness over baselines, with gains sustained through one- to five-minute generation.

  • Benchmarks: A2RD is evaluated on VBench-Long with 40 prompts expanded into eight continuous segments for approximately one-minute videos, and on LVbench-C with 24- and 40-scene videos lasting three and five minutes.The experiments cover both single-scene and multi-scene long-video generation.
  • Single-Scene Results: A2RD reaches 0.90 narrative coherence on one-minute VBench-Long videos, exceeding Naive-AR’s 0.75 and the lower scores of ViMax and VideoMemory.ViMax scores 0.69 and VideoMemory 0.67, while A2RD improves consistency across characters and environments.
  • Single-Scene Results: 0.9935 motion consistency substantially surpasses baselines, while character consistency reaches 0.74 versus 0.57/0.56 and environment consistency reaches 0.84 versus VideoMemory’s 0.73.A2RD-Par also achieves 0.81 narrative coherence while supporting parallel generation.
  • Multi-Scene Results: 0.38 baseline character consistency and 0.40 baseline environment consistency at most expose severe degradation on LVbench-C’s three- and five-minute multi-scene settings.Baselines also show lower semantic alignment at five minutes than at three or one minute, highlighting extended-horizon prompt-fidelity challenges.
  • Scaling Baselines: 0.67 average consistency is achieved by Naive-AR after best-of-N scaling, versus 0.61 before scaling; VideoMemory improves from 0.65 to 0.71.Scaling samples the same number of videos and frames per segment as A2RD, but narrative coherence does not consistently improve.
  • Human Results: 4.68/5.00 is A2RD’s average human score across six criteria, exceeding VideoMemory’s 3.93 and including 4.89 character consistency, 4.75 narrative coherence, and 4.34 transition smoothness.The study uses 40 VBench-Long samples rated by seven evaluators on a five-point Likert scale.

6. Analysis

The analyses show that A2RD’s memory, test-time scaling, and adaptive generation components are essential for consistency, while its gains generalize across diffusion backbones and remain strong over ultra-long horizons.

  • Ablation results: Ablations show that MVMem, HITS, MAPO, global references, and adaptive segment generation each materially contribute to A2RD’s consistency.Removing HITS lowers narrative scores from 0.90 to 0.74 and character scores from 0.74 to 0.68; removing global references lowers environment consistency from 0.84 to 0.79.
  • Backbone generalization: A2RD improves over Naive-AR on both LTX-Video and Wan 2.2 without costly retraining.On LTX-Video, narrative coherence rises from 0.59 to 0.79 and character consistency from 0.50 to 0.70; Wan 2.2 shows similar gains.
  • Long-horizon consistency: A2RD maintains higher consistency over extended LVbench-C horizons, while baselines decline monotonically as the number of scenes grows.Environment consistency is the most challenging dimension overall, and character performance varies according to the availability of dedicated memory mechanisms.
  • Long-horizon consistency: The MLLM-Judge flags obvious consistency errors with 80% agreement, though it may miss subtle inconsistencies.Its overall trend agrees with automatic and human evaluations showing that A2RD significantly outperforms baselines.
  • Ultra-long video generation: 90.5% character, 84.0% environment, and 91.5% object consistency are achieved by A2RD on ten 10-minute LVbench-C scenarios.These averages support its capability to generate coherent ultra-long videos.

7. Conclusions · A. Terminologies and Summary of Major Implemented Functions

A2RD is presented as an agentic autoregressive architecture addressing long-video temporal consistency and narrative coherence through multimodal memory, adaptive generation, and hierarchical self-improvement. The paper also introduces LVBench-C, reports state-of-the-art results across 1–10 minute videos, and documents computational, evaluation, model-capability, and rubric limitations alongside terminology and function summaries.

  • 7. Conclusions: A2RD decouples creative synthesis from consistency enforcement to address temporal consistency and narrative coherence in long-video synthesis.
  • 7. Conclusions: Its three components are multimodal video memory, adaptive generation for narrative progression and consistency, and hierarchical self-improvement for frames and segments.
  • 7. Conclusions: A2RD introduces LVBench-C, which stress-tests long-horizon consistency through cyclical appearance with optional state evolutions.
  • 7. Conclusions: Across 1–10 minute videos, A2RD sets a new state-of-the-art for narrative coherence and visual consistency against existing baselines.
  • Limitations: A2RD incurs more computational overhead than the experimented baselines, although its active reasoning is described as incurring insignificant costs.
  • Limitations: Human agreement scores are not reported because some videos had exactly two reviewers and long-form evaluation is inherently subjective.
  • Limitations: A2RD requires strong instruction-following and visual-reasoning capabilities from its component models, while its verification rubrics may not generalize across styles, cultures, or domains.
  • A. Terminologies and Summary of Major Implemented Functions: The paper includes a glossary of key terms and a summary of the main MLLM and synthesis functions implemented in A2RD.

B. Additional Analysis · B.1. General Video Quality Assessments

A2RD is evaluated on six VBench metrics for general video quality and performs competitively overall. It achieves the best Subject Consistency score, while A2RD-Par leads Background Consistency and Temporal Flickering.

  • B.1. General Video Quality Assessments: A2RD is assessed across six VBench metrics measuring general video quality.The evaluation uses the metrics reported in Table 8.
  • B.1. General Video Quality Assessments: A2RD performs competitively across all six general video quality metrics.The results indicate broad performance rather than leadership on only one measure.
  • B.1. General Video Quality Assessments: A2RD achieves the best score in Subject Consistency.This is the strongest single-metric result attributed to A2RD in the assessment.
  • B.1. General Video Quality Assessments: A2RD ranks second in Background Consistency, Aesthetic Quality, and Imaging Quality.These results place A2RD near the top on three additional quality dimensions.
  • B.1. General Video Quality Assessments: A2RD-Par obtains the highest scores in Background Consistency and Temporal Flickering.The parallel variant outperforms A2RD on these two reported metrics.
  • B.1. General Video Quality Assessments: The results indicate that A2RD does not sacrifice visual fidelity for temporal consistency.This conclusion follows from its competitive general-quality performance alongside the temporal-consistency focus.

B.2. Methodology Design Analysis: Why Does A2RD Work? … B.2.3. How A2RD Maintains Long-Distance Motion Continuity (MLLMimg

A2RD’s effectiveness is analyzed through adaptive segment generation, frame-video co-optimization, MVMem textual states, MLLM-Judge criteria, MAPO, and retrieval for long-distance motion continuity. The analyses show how adaptive mode selection, end-frame correction, and retrieval preserve narrative, visual, and motion consistency across temporally distant scenes.

  • B.2. Methodology Design Analysis: Why Does A2RD Work?: A2RD’s methodology analysis examines Adaptive Segment Generation, Frame-Video Co-Optimization, MVMem’s Textual States, MLLM-Judge criteria, and MAPO.Keyframes and other MLLM-Judge criteria are omitted because they have been widely studied.
  • B.2.1. Adaptive Segment Generation Mechanism: Adaptive Segment Generation balances natural narrative progression with consistency enforcement by selecting generation modes according to scene context and established environments.In the illustrated LVbench-C example, A2RD selects Interpolation mode when the next scene occurs in a known Cobblestone Alleyway.
  • B.2.1. Adaptive Segment Generation Mechanism: > 85% agreement with human experts is achieved by the MLLM for generation-mode determination.Most errors involve classifying extrapolation as interpolation, but these mistakes are largely benign because the image model can sometimes maintain highly consistent environments.
  • B.2.2. Why Co-Optimization in Extrapolation Mode Is Crucial (Section 3.3): End-frame co-optimization corrects inconsistencies that persist when extrapolation uses an unoptimized scene endpoint.Figure 9 contrasts missing blue-scarf and incorrect-character-identity errors without co-optimization against their natural correction with it.
  • B.2.2. Why Co-Optimization in Extrapolation Mode Is Crucial (Section 3.3): Without end-frame optimization, directly using Scene 18’s endpoint and optimizing only the bridging prompt can preserve a missing blue scarf across subsequent scenes.Prompt specification may sometimes produce a similar scarf, but the cited analysis identifies this as an inconsistency that co-optimization addresses.
  • retr in Equation (5)): retr is presented as a key contribution for bridging long-distance motion continuity, which prior works had not addressed according to the paper.The mechanism is illustrated through a multi-shot Scene 6 and later return to the red sports car.
  • retr in Equation (5)): MVMem’s Textual States explicitly enforce key details in prompts, leading to highly consistent outcomes.The marathon example in Figure 11 demonstrates the role of textual states in consistency.
  • retr in Equation (5)): A2RD retrieves the correct stored end-of-shot frame and uses it as a later scene’s begin frame, preserving visual and motion continuity across a large temporal gap.In the example, Shot 1 of Scene 6 is retrieved for Scene 12 because only segment begin and end frames are otherwise saved.

B.2.4. The Role of MVMem’s Textual States · B.2.5. Why Do We Need Those MLLM-Judge Criteria? · B.2.6. Memory-Augmented Prompt Optimization (MAPO)

The sections show how A2RD uses textual memory states, targeted MLLM-Judge criteria, and MAPO to preserve details, prevent continuity failures, and improve prompts. Together, these mechanisms address entity and environment consistency, physical and spatial plausibility, narrative progression, and actionable prompt refinement.

  • B.2.4. The Role of MVMem’s Textual States: Textual states explicitly retrieve and enforce missing entity and environment details in prompts, alongside reference frames and global anchors, preserving consistency across scenes.Without these states, important details such as Racing Bib 402 and rainbow elements can disappear during synthesis.
  • B.2.4. The Role of MVMem’s Textual States: A2RD’s MLLM-Judge identifies spatial contradictions even when a frame otherwise captures prompt elements and maintains narrative coherence.Its analysis flags a vehicle-orientation contradiction against the established environment, illustrating why spatial checks complement general consistency judgments.
  • B.2.5. Why Do We Need Those MLLM-Judge Criteria?: Physical Plausibility is needed because consistency enforcement can produce awkward frames whose use as video starting frames causes hallucinations at the beginning.Figure 12 illustrates how physical implausibility can propagate into the resulting video.
  • B.2.5. Why Do We Need Those MLLM-Judge Criteria?: Spatial Logicalness maintains continuity across segments, since a 90-degree car rotation between scenes can create discontinuity and hallucination artifacts during interpolation.The failure occurs despite Scene 3 achieving perfect scores on other consistency and basic quality metrics.
  • B.2.5. Why Do We Need Those MLLM-Judge Criteria?: Narrative Progression balances story development against consistency, because a storyboard can remain highly consistent in entities and environments while lacking meaningful narrative.This criterion applies at both frame and video levels.
  • B.2.6. Memory-Augmented Prompt Optimization (MAPO): MAPO diagnoses prompt weaknesses such as abstract camera references and missing spatial ordering of background elements.The baker example uses validation feedback to identify concrete areas for improvement.
  • B.2.6. Memory-Augmented Prompt Optimization (MAPO): MAPO retrieves 10 positive and 5 negative cases, synthesizes 12 lessons, and applies them to refine prompts with concrete anchors, tactile actions, hand positions, and linear environmental ordering.The retrieved lessons replace abstract references with physical descriptions and improve specification of actions and scene layout.

B.3. Error Analysis

The error analysis identifies environment consistency as the most challenging dimension, with failures involving physical-layout approximation and hallucinated elements. Entity inconsistencies can also evade the MLLM-Judge, motivating more accurate judges and stronger reasoning models.

  • Environment Inconsistencies: Environment consistency is the most challenging dimension because image synthesis struggles to control layouts and physical arrangements.This limitation is illustrated by representative failure cases flagged by the MLLM-Judge.
  • Environment Inconsistencies: Environment failures include visually similar but physically different scenes and hallucinated objects, such as a wooden table abruptly appearing during coffee service.These errors reflect approximated physical layouts or hallucinated physical elements.
  • Entity Inconsistencies: Entity inconsistency occurs when the same checkered shirt is reproduced with a slightly different color than the reference.The MLLM-Judge occasionally misses such minor inconsistencies, allowing them to go unnoticed during self-refinement.
  • Future Directions: Addressing these failures likely requires more accurate MLLM judges and stronger reasoning foundation models with physics-aware control.The analysis identifies improved foundation-model reasoning and physics-aware control as promising future directions.

B.4. Test-Time Self-Improvement Analysis … C. Implementation Details

The analysis shows that HITS improves frame- and video-level consistency across refinement iterations, while the latency analysis details A2RD’s computational costs, parallelization, and practical trade-offs. The authors also disclose using AI-assisted tools for coding and writing support.

  • B.4. Test-Time Self-Improvement Analysis: HITS refinement improves frame-level scores, raising spatial logicalness from 7.083 →8.708 and the overall average from 8.741 →9.407 after two iterations.Physical plausibility also increases from 8.583 →8.792.
  • B.4. Test-Time Self-Improvement Analysis: At the video level, inter-consistency improves steadily across iterations, with environment consistency showing the largest gain from 7.430 →8.795.The scores were recorded across two refinement iterations on all benchmarks using calibrated MLLM-Judge criteria.
  • B.5. Estimated Latency Analysis: Naive-AR incurs approximately NL_M+NL_V latency, whereas Naive-Par incurs NL_M+(N+1)L_I+L_V under ideal hardware assumptions.Here, L_M, L_I, and L_V denote single MLLM, TI2I, and TI2V call latencies, respectively.
  • B.5.1. Baseline Latency: MovieAgent decomposes scripts hierarchically through script breakdown, scene planning, and shot creation, issuing one LLM call per unit before per-shot image and video synthesis.VideoMemory instead uses StoryboardAgent once, followed by sequential shot processing with dynamic memory.
  • B.5.2. A2RD Latency: A2RD initializes multimodal memory with maximum latency L_init ≈2L_M+|R|L_I and precomputes retrieval and generation-mode decisions with one batch MLLM call.The initialization uses two batch MLLM calls and can synthesize global references for unique entities.
  • B.5.2. A2RD Latency: A2RD-Par sets N_interp=N and k_v=1, fully parallelizing video synthesis so its latency does not scale with N and incurs L_M+L_V.A2RD’s total latency is defined as L_A2RD=L_init+L_pre+L_frame+L_video.
  • B.5.2. A2RD Latency: With k_v=k_f=3, A2RD adds roughly 12 MLLM calls, 2 TI2I calls, and 2 TI2V calls per segment relative to VideoMemory.Observed latencies are under 10 seconds per MLLM call, 30 seconds per TI2I call, and under 120 seconds per TI2V call.
  • B.6. Use of AI Assistants: The authors used ChatGPT, Gemini, and Claude for coding and writing support, while retaining sole responsibility for substantive content, analysis, and conclusions.This disclosure concerns the paper’s use of AI assistants.

C.1. Automatic Metrics’ Implementations … C.1.3. Prompt for Grouping Scenes for Inter-Shot Metrics

The paper specifies implementations for automatic metrics and prompts for candidate selection, narrative-coherence scoring, and scene grouping. These procedures combine embedding-based similarity, boundary-motion evaluation, structured MLLM judgments, and JSON outputs.

  • C.1. Automatic Metrics’ Implementations: Semantic Alignment uses ViCLIP to measure cosine similarity between video and scene-description embeddings.This metric is implemented through multimodal embedding similarity.
  • C.1. Automatic Metrics’ Implementations: Inter-shot character and environment consistency groups scenes by shared characters, states, environments, and states before comparing region embeddings.YOLOv8 segments character and background regions, DINOv3 extracts embeddings, and pairwise cosine similarities are averaged within groups.
  • C.1. Automatic Metrics’ Implementations: Inter-Shot Motions evaluates transitions using the preceding segment’s last 10 frames and the subsequent segment’s first 10 frames with VBench motion_smoothness.Each scene is paired with its previous continuous scene when one exists.
  • C.1.1. Best-of-N Baseline’s Prompts: The Best-of-N image prompt selects one candidate generated from the same scene description and reference entities using scene faithfulness, visual quality, and entity consistency.The evaluator must choose holistically and return only JSON containing the best candidate and a brief reason.
  • C.1.1. Best-of-N Baseline’s Prompts: The Best-of-N video prompt selects one clip from candidates sharing a prompt and previous clip using prompt faithfulness, visual quality, and motion naturalness.Motion naturalness requires smooth, physically plausible continuation from the previous clip, with JSON-only output.
  • C.1.2. Prompt for Narrative Coherence Evaluation: Videos, characters, or environments that are nonsensical, imperceptible, or incomprehensible receive a narrative-coherence score capped at 0.2.The evaluator outputs a JSON object containing narrative_coherence and reasoning.
  • C.1.3. Prompt for Grouping Scenes for Inter-Shot Metrics: The scene-grouping prompt returns JSON arrays for shared backgrounds, character appearances, and object appearances using 0-based scene indices.Groups capture consistent locations, character features, and object properties such as color, shape, and material.

C.2. Human Metrics’ Scoring Guidelines

The human evaluation presents randomized, anonymized multi-segment videos for review against reference materials and scene descriptions, with ratings collected on six criteria using a 5-point Likert scale.

  • Evaluation procedure: Human evaluation participants review a randomly sampled, randomized, anonymized subset of generated videos from all methods.The study takes approximately 30 minutes, and each video contains multiple 8-second segments.
  • Scoring guidelines: Participants watch each video, review reference images and scene descriptions, and rate six criteria using a 5-point Likert scale.

Evaluation Criteria … F.2. Baseline Examples

The paper evaluates video quality through consistency, smoothness, coherence, and reference fidelity, while illustrating LVBench-C’s evolving entities and environments and detailing prompts for retrieval, synthesis, refinement, and judging. Its examples emphasize maintaining visual, spatial, temporal, and narrative continuity across scenes and segments.

  • Evaluation Criteria: Evaluation criteria cover character, object, environment, transition, narrative, and reference consistency.These criteria assess appearance preservation, natural transitions, logical story progression, and fidelity to references.
  • D. LVbench-C: Examples: LVBench-C examples test non-consecutive object, character, and environment state evolution across long scenarios.Examples include a Dutch oven that disappears and reappears, characters that return with evolved states, and locations evolving through a storm sequence.
  • E. Methodology Prompts: Multimodal memory initialization identifies entities, environments, dependencies, and comprehensive spatial references before synthesis.Entity descriptions remain minimal for state evolution, while environment descriptions capture necessary static details globally.
  • E. Methodology Prompts: Adaptive segment generation groups scenes by shared location, temporal continuity, and direct action flow, while separating time jumps and fundamental environment changes.Moving environments permit continuous framing changes, whereas significant framing changes in static environments begin new segments.
  • E. Methodology Prompts: Reference retrieval and frame prompting preserve identities, environments, camera relations, and spatial states across scenes.The prompts retrieve relevant anchors and prior scenes, then refine frame prompts using memory and reference images.
  • F.1. A2RD Examples: A2RD’s storyboard process synthesizes frames and video segments from refined prompts while maintaining strict visual consistency and narrative progression.Frame synthesis uses reference images, and video synthesis uses generated prompts to produce each segment.
  • F. Full Storyboard Examples: Video-segment prompting continues entity, environment, motion, camera, and narrative states from previous scenes without reversing camera direction.Prompts can enforce exact ending frames or allow natural completion depending on whether an ending frame is provided.
  • F.2. Baseline Examples: Frame judging scores instruction following and physical plausibility on a 1-10 scale.The judge examines whether frames capture prompt details, lighting, environmental conditions, and physically plausible entities.
Loading 2605.06924v1…