VChain: Chain-of-Visual-Thought for Reasoning in Video Generation

• Video generators often produce smooth clips but miss meaningful state transitions and causal steps, while multimodal models can infer likely progressions between visual states.The introduction contrasts smooth deformations that omit consequences, such as a cup's impact and resulting splash, with multimodal models' strength in multi-step reasoning.Source: §1 · p. 2
• VChain turns prompts into sparse Visual Thoughts—causally important keyframes from a multimodal reasoning pipeline—and uses them to adapt a pretrained generator at those critical states.The keyframes serve as a blueprint for temporal structure and guide intended visual-state transitions through focused inference-time tuning.Source: §1 · p. 2
• VChain synthesizes supervision during inference from a few keyframes, requiring no external annotations, curated data, or retrieval and adding minimal overhead.Its keyframe-only tuning uses limited iterations and is presented as faster and more practical for deployment than tuning on video data.Source: §1 · pp. 2–3
• VChain improves dynamic fidelity on complex, multi-step tasks, producing videos with more logical consequences, smooth transitions, and coherent visual narratives.The framework treats multimodal models as reasoning modules that complement video generators rather than replace them.Source: §1 · p. 2

2.1 Video Generation

• Despite progress in video generation, most methods remain limited to surface-level coherence and miss causal dynamics, physical interactions, and meaningful state transitions (Zheng et al., 2025; Yue et al., 2025).The cited advances include diffusion models, variational autoencoder-based compression, and transformer backbones.Source: §2.1 · p. 3

2.2 Multimodal Models for Understanding and Generation

• LLMs and multimodal models, including GPT-4, Gemini, and GPT-4o, show strong vision-language abilities and can reason about scenes and generate grounded visual content (OpenAI, 2023; Team et al., 2023; Hurst et al., 2024).Their capabilities include instruction-following, visual question answering, and interactive reasoning.Source: §2.2 · p. 3
• Prior work uses multimodal models for image synthesis or coarse-layout guidance, while other approaches treat them as prompt interpreters or planners, or require dense retraining and architectural changes (Zhou et al., 2024; Lian et al., 2023a; Lian et al., 2023b; Li et al., 2023; Qu et al., 2023; Wang et al., 2026a; Pan et al., 2025; Lin et al., 2025).VChain instead uses off-the-shelf multimodal models and pretrained video generators, injecting sparse visual reasoning signals at inference time with minimal overhead.Source: §2.2 · p. 3

3.1 Preliminaries

• The framework builds on Wan’s video-generation model and uses LoRA, which freezes original model weights while adding trainable low-rank updates (Hu et al., 2022).Wan uses a video diffusion transformer and flow matching in the Wan-VAE latent space.Source: §3.1 · p. 4

3 The VChain Framework

• VChain has three stages: infer sparse visual thoughts, adapt a pre-trained generator with them via LoRA, and sample a video using the thoughts and tuned generator.The framework overview identifies these stages as Visual Thought Reasoning, Sparse Inference-Time Visual-State Adaptation, and Video Sampling.Source: §3 · p. 3, Figure 2 · p. 5

3.4 Video Sampling

• Video sampling concatenates all textual thoughts into one prompt for the fine-tuned generator, combining the inferred event sequence with its adapted capabilities.The final video is generated using the composite prompt and the tuned model.Source: Figure 2 · p. 5, §3.4 · p. 6

3.2 Visual Thought Reasoning

• GPT-4o infers a prompt’s likely outcome, then iteratively generates keyframes and paired textual thoughts until the consequence is captured.The resulting sequence represents the temporal evolution implied by the prompt and can provide causal consistency without requiring users to specify the consequences themselves.Source: §3.2 · pp. 4–5

3.3 Sparse Inference-Time Visual-State Adaptation

• Sparse adaptation treats each visual thought as a one-frame video paired with its textual thought, providing focused, image-only supervision without extra datasets or labels.These internally generated keyframes anchor important state changes and are used to tune the video diffusion model with a flow-matching objective.Source: §3.3 · pp. 5–6

4.1 Experimental Setup

• The experiments use 20 diverse test scenarios for human evaluation and quantitative comparisons.The main experiments use GPT-family models for visual thought reasoning and the pretrained Wan2.1-T2V-1.3B generator.Source: §4.1 · p. 6

4.2 Comparison Methods

• Ablations compare the full framework with variants omitting visual thoughts or sparse tuning, while baselines include original T2V and GPT-based prompt augmentation.The full VChain framework combines Visual Thought Reasoning with Sparse Inference-Time Visual-State Adaptation.Source: §4.2 · p. 6

4.4 Qualitative Comparisons

• In the bowling comparison, VChain depicts a coherent ball impact and plausible pin knockdown, unlike static T2V output and the erratic, unstable motion of prompt augmentation.The VChain sequence also preserves object geometry and material appearance across frames.Source: Figure 3 · p. 7, §4.4 · p. 8

4.3 Quantitative Comparisons

• Human evaluations assess frame quality, temporal quality, text alignment, physics, commonsense, and causal reasoning, with 32 evaluators rating each video from 1 to 5.Outputs were shown in randomized order, and scores were averaged and normalized to percentages.Source: §4.3 · p. 7
• VChain scores comparably or slightly better than the pretrained generator and other baselines on VBench, and consistently outperforms them on physics, commonsense, and causal reasoning.The reported results indicate that its reasoning gains do not compromise basic visual quality and can bring modest improvements.Source: §4.3 · p. 7

4.4 Qualitative Comparisons

• In the ball-catching ablation, removing visual thoughts misses the first-person catch viewpoint, while removing sparse tuning causes warping; the full method produces the most coherent interaction.Figure 4 shows that removing either component degrades video synthesis quality.Source: Figure 4 · p. 8, §4.4 · p. 8

5.1 Limitations of Visual Thought Generation

• gpt-image-1 can produce over-saturated, over-smooth frames whose feedback accumulation adds a yellow cast and reduces later-frame photorealism; each keyframe also requires two API calls.Total calls scale linearly and token consumption quadratically, though inference typically uses only 3–6 images and practical overhead remains modest.Source: §5.1 · pp. 8–9
• GPT-4o’s closed-source nature limits customization and full reproducibility; integrating capable open-source MLLMs could improve accessibility and reproducibility.Source: §5.1 · p. 9

5.2 Limitations of Sparse Inference-Time Visual-State Adaptation

• Sparse keyframe tuning trades motion dynamics against reasoning strength: over-optimization may reduce dynamics, while under-optimization weakens injected signals toward the untuned baseline.The pre-trained video generator is fine-tuned using several keyframes, called Visual Thoughts, as supervision.Source: §5.2 · p. 9
• Despite incomplete capture of video dynamics, semantic-alignment and causal-coherence improvements generally outweigh the loss, and carefully selected keyframes can guide adaptation to new prompts or scenarios.Sparse tuning focuses capacity on critical transitions and avoids dense-video preparation, reducing data and computational overhead for inference-time integration.Source: §5.2 · p. 9

Potential Risks

• The authors advocate responsible and ethical use of generative models.VChain is intended for controlled research and creative exploration.Source: §6 · p. 9
• VChain’s greater realism could enable disinformation, deepfakes, or fabricated evidence.The authors say it is not intended for sensitive or adversarial settings.Source: §6 · p. 9
• VChain may propagate biases from its multimodal models and pretrained generators into more coherent narratives, reinforcing stereotypes or exclusion.Users should exercise caution because generated outputs may inherit and amplify safety concerns and biases.Source: §6 · p. 9

7 Conclusion

• VChain represents scenarios as sparse keyframes of critical intermediate states inferred by large multimodal models, injecting causal and commonsense reasoning signals at inference time.This lets video generators model meaningful state transitions without dense annotations or costly retraining.Source: §7 · p. 9
• Experiments on complex, multi-step scenarios show that VChain substantially improves generated videos’ coherence, causal consistency, and rationality while maintaining efficiency and visual quality.Source: §7 · p. 9
• VChain combines multimodal models’ reasoning capabilities with video generators’ rendering and motion priors as a step toward bridging reasoning and generation.Source: §7 · p. 9

Appendices

• Appendix B provides qualitative results, Appendix C discusses video-generation reasoning paradigms and VChain’s place in their taxonomy, and a demo video is available.Appendix A contains additional implementation details.Source: Appendices · p. 13

A.1 Implementation Details of Visual Thought Reasoning

• GPT-4o reasons from the user prompt about spatial layout and causal consequences, and LangChain structures its output into context-frame, concise-prompt, and consequences fields.The context frame is sent to gpt-image-1 to generate the first image.Source: §A.1 · p. 13
• For each later keyframe, GPT-4o receives the stitched prior images, user prompt, and inferred consequences, then predicts an edit instruction and whether the sequence has reached a terminal state.The editing instruction guides gpt-image-1; generation repeats until the terminal-state flag is true.Source: §A.1 · p. 13
• The pipeline stores keyframe captions and reasoning chains in JSON, then creates CSV image-text pairs, pairing the first image with the concise prompt and later images with their keyframe captions.These pairs are used to fine-tune the video generation model.Source: §A.1 · p. 13

A.2 Time Cost

• Table 2 summarizes average runtime by VChain stage and provides a breakdown of overall computational cost.The supplied passage does not report runtime values.Source: §A.2 · p. 13

A.3 Implementation Details of Sparse Inference-Time Visual-State Adaptation

• The main experiments use Wan2.1-T2V-1.3B (Wan et al., 2025), with learning rate 1e −4, train_lora_rank 16, and train_lora_alpha 16.Source: §A.3 · p. 13

A.4 Details of Test Cases

• Twenty physically grounded test cases support human and quantitative evaluation of causal reasoning for coherent outcomes.Scenarios include falling objects, mixing liquids or paints, and gradual changes such as rotting or rusting.Source: §A.4 · pp. 13–14
• Figure 5 illustrates the inferred causal progression across keyframes for sulfuric acid poured onto a wooden table.The figure presents the pipeline’s reasoned Visual Thoughts for this prompt.Source: Figure 5 · p. 14
• Recursive GPT image generation can compound saturation and smoothness artifacts, shifting colors and reducing later-frame photorealism.The examples show a yellow cast as generated images are reused as input for subsequent keyframes.Source: Figure 6 · p. 14

B Additional Qualitative Results

• Additional qualitative examples illustrate saturation limitations in Visual Thought Generation.Further comparisons include egg-fall, pillow, rocket-feather, cup, and egg-pillow cases.Source: B Additional Qualitative Results · p. 14, Figure 13 · p. 21, Figure 14 · p. 21

C Discussion: Evolution of Reasoning Paradigms

• The discussion situates VChain within the evolution of reasoning paradigms and their application to video generation and generative tasks.Source: C Discussion: Evolution of Reasoning… · p. 24

C.1 From Textual to Visual Reasoning in the Era of Large Foundation Model

• Textual reasoning includes explicit CoT and ToT and implicit methods, but transferring linguistic reasoning to video leaves physical, spatial, and temporal transitions difficult to execute faithfully.The passage characterizes this as a semantic gap between describing causal sequences in language and generating their visual outcomes.Source: §C.1 · p. 24
• VChain addresses that gap by shifting reasoning from textual or latent linguistic space to explicitly grounded visual states for video generation.Source: §C.1 · p. 24

C.2 Taxonomy of Reasoning in Video Generation

• Video-generation reasoning spans implicit end-to-end learning, text-guided planning, and MLLM-aided feature-level methods, which respectively risk causal hallucinations, struggle with fine-grained transitions, or require dense retraining and structural changes.Source: §C.2 · p. 24
• VChain introduces Visual Thought Reasoning, externalizing reasoning as sparse visual keyframes that guide inference-time adaptation to physically grounded states.The approach is termed Sparse Inference-Time Visual-State Adaptation.Source: §C.2 · p. 24

Key Messages and Future Outlook

• Visual chain-of-thought is necessary because only visual thoughts provide spatial and material constraints that can override a generator’s incorrect physical priors.Textual reasoning supplies a logical blueprint, while visual thoughts translate symbolic logic into pixel-level consistency.Source: §C.3 · p. 25
• VChain establishes a modular paradigm in which a powerful MLLM Reasoner guides a specialized video Renderer.The examples given are GPT-4o and Gemini as Reasoners and a video diffusion transformer as Renderer.Source: §C.3 · p. 25
• Emerging video-generator reasoning may blur the Reasoner–Renderer boundary, but VChain remains useful for complex multi-step causal consistency while MLLM reasoning outpaces video generators.The passage cites emerging spatial-reasoning research (Wiedemer et al., 2025; Wang et al., 2026b) and says the reasoning gap may persist for a while.Source: §C.3 · p. 25