Source-linked AI summary
Direct, Parallel, or Sequential? A Comparative Study of Training-Free Multi-Subject Image-to-Video Generation
Yanliang Qi, Kexi Chen, Muchao Ye, Haomiao Ni
TL;DR
Multi-subject I2V must preserve several subject identities, assign distinct motions, and maintain coherent interactions, but the capabilities of frozen pretrained models in this setting remain underexplored. The paper systematically compares direct, parallel, and sequential training-free paradigms under a unified protocol. It finds no universally optimal strategy: sequential generation favors compositional consistency, direct generation favors efficiency and smooth motion, and parallel generation favors modularity without consistently improving final metrics.
Problem
Multi-subject I2V remains underexplored because frozen pretrained models must preserve multiple appearances, follow individual motions, and maintain coherent inter-subject relationships.
Method
The paper systematically compares direct, parallel, and sequential training-free multi-subject I2V generation under matched backbones and sampling settings.
Results
No paradigm is universally optimal: sequential generation achieves the strongest subject and background consistency, while direct generation produces the smoothest motion and parallel generation does not consistently improve final quantitative results.
Takeaways & Limitations
Direct generation favors efficiency and temporal smoothness, parallel generation favors modularity, and sequential generation favors compositional consistency at higher computational cost.
Takeaways & Limitations
Parallel generation struggles with strongly interacting subjects, while sequential generation is sensitive to insertion order and accumulated errors.
Abstract
from arXiv · showhide
Text-conditioned image-to-video (I2V) generation has advanced rapidly, yet generating videos with multiple subjects remains challenging. A model must simultaneously preserve the appearance of each subject, assign distinct motions, and maintain coherent spatial and temporal interactions. This paper presents a systematic study of three representative paradigms for training-free multi-subject I2V generation: direct, parallel, and sequential generation. Direct generation applies a pretrained I2V model to the complete reference image and prompt, requiring all subjects and motions to be synthesized jointly. Parallel and sequential generation instead decompose the reference image and prompt into subject-specific visual and textual conditions. Parallel generation synthesizes each subject independently and subsequently composes the resulting videos, reducing the complexity of each generation step at the cost of weaker inter-subject context. Sequential generation first synthesizes a background video and then progressively introduces individual subjects. This preserves accumulated scene context but introduces sensitivity to subject ordering and error propagation. We empirically evaluate the three paradigms across diverse multi-subject scenes, comparing appearance preservation, motion fidelity, temporal consistency, and inter-subject coherence, while also characterizing their distinct failure modes. Our findings reveal the strengths and limitations of each paradigm and offer practical insights for designing controllable multi-subject video generation systems.
1 Introduction
Multi-subject I2V remains challenging because frozen pretrained models must preserve distinct appearances, follow individual motions, and maintain coherent interactions. The paper compares direct, parallel, and sequential training-free strategies, finding different trade-offs in synthesis complexity, context preservation, and efficiency.
- Multi-subject I2V requires preserving multiple appearances, following distinct motions, and maintaining coherent spatial and temporal interactions.
- Direct generation synthesizes the complete scene jointly, preserving image–prompt context but risking entangled appearance, motion, and interaction artifacts.
- Parallel generation decomposes inputs into subject-specific pairs, generates videos independently, and merges them post hoc, reducing per-subject burden while weakening shared context.
- Sequential generation introduces subjects progressively while conditioning on previously generated video context, preserving evolving context but increasing inference time and error propagation.
- No paradigm is universally superior: direct generation is simple and efficient, parallel generation reduces per-subject complexity, and sequential generation preserves accumulated context.
2 Related Work
Prior I2V work distinguishes stochastic generation from conditional generation based on whether an external motion condition is used. This paper focuses on text-conditioned I2V and builds on representative text-driven generators.
- I2V generation can be categorized as stochastic, using only an image, or conditional, using an image and an external condition.
- The paper primarily studies text-conditioned I2V generation, where the external condition is a text prompt.
- MAGE uses a motion-anchor structure and three-dimensional axial transformers to capture appearance–motion-aligned representations.
3 Methodology
The methodology compares direct generation with two decomposed, training-free paradigms: parallel composition and sequential progressive injection. These approaches differ in whether subjects are synthesized jointly, independently and then composed, or introduced into an evolving context.
- Paradigm Overview: The study compares direct, parallel, and sequential training-free paradigms for multi-subject I2V generation.The goal is to preserve subject appearance, follow motion prompts, and maintain coherent spatial and temporal relationships.
- Sequential Generation: Sequential generation inserts subjects one by one into a shared evolving video context and then applies temporal refinement.The procedure preserves accumulated context and subject layout, while remaining sensitive to insertion order and propagated errors.
- Direct Generation: Direct generation applies the pretrained I2V model to the full image and prompt in one pass.It preserves complete image–prompt context and the original spatial layout, but jointly synthesizes all subjects, motions, and interactions.
- Input Decomposition: The shared decomposition first derives scene- and subject-level prompts, then uses detection, segmentation, masking, inpainting, and cropping to construct corresponding image components.These decomposed text-image pairs serve as common inputs for parallel and sequential generation.
- Parallel Generation: Parallel generation decomposes text and images into background- and subject-level pairs, generates component videos independently, and composes them afterward.Each I2V call handles one subject or the background, reducing per-call synthesis burden; independently generated videos may lack compatible interactions and occlusions.
4 Experiments
The experiments compare direct, parallel, and sequential training-free paradigms under a unified protocol, using two pretrained backbones and five VBench-style metrics. Results show a consistent trade-off: sequential generation favors subject and background consistency, direct generation favors motion smoothness, and parallel generation does not consistently improve final metrics.
- Experimental setup: The study evaluates direct, parallel, and sequential generation with two pretrained backbones under matched datasets, metrics, and output settings.The backbones are I2VGen-XL and DynamiCrafter; outputs are standardized to 512 × 512 resolution, 16 frames, and 8 FPS.
- Experimental setup: Five VBench-style metrics measure subject consistency, background consistency, motion smoothness, aesthetic quality, and imaging quality.All scores are reported as percentages, with higher values indicating better performance.
- Quantitative results: Sequential generation achieves the highest subject consistency and background consistency across matched backbone and sampling configurations.Progressive insertion preserves accumulated subject and scene context, but repeated composition and refinement reduce motion smoothness relative to direct generation.
- Quantitative results: Direct generation consistently provides the highest motion smoothness, while frame-level aesthetic and imaging quality depend more strongly on the pretrained backbone.Direct generation retains stronger AQ and IQ with I2VGen-XL, whereas sequential generation performs best with DynamiCrafter in most settings.
- Paradigm analysis: Parallel generation reduces per-call complexity but lacks shared temporal context, so composition can introduce inconsistency and does not consistently improve final metrics.Independent subject streams may contain appearance or motion errors that are inherited by the final composition.
- Failure modes and efficiency: The paradigms exhibit distinct qualitative and computational failure modes: direct generation can entangle subjects, parallel generation can confuse identities, and sequential generation can accumulate intermediate errors.Sequential generation also requires repeated composition and refinement, increasing inference time and GPU-memory demands compared with direct generation.
5 Conclusion and Discussion
The comparison reveals a consistent division of strengths among the three paradigms rather than a universally optimal approach. Sequential generation favors subject and background consistency, while direct generation favors smooth motion and efficiency, with quality outcomes depending partly on the backbone.
- Sequential generation achieves the strongest subject and background consistency among the compared paradigms.
- Direct generation consistently produces the smoothest motion and requires substantially less inference time.
- Frame-level aesthetic and imaging quality depend on the backbone, favoring sequential generation with DynamiCrafter but direct generation with I2VGen-XL.
- Parallel generation offers modular decomposition, but independent generation and post-hoc composition do not consistently improve final quantitative results.
- The study is limited to a relatively small synthetic benchmark, two pretrained I2V backbones, and five automatic evaluation metrics.