Source-linked AI summary
DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models
Huanran Hu, Zihui Ren, Dingyi Yang, Zhinan Song, Guozheng Wu, Tiezheng Ge, Qin Jin
TL;DR
Video diversity evaluations often use global scores that do not show which factors collapse, limiting diagnosis of repeated outputs from the same prompt. DiVid measures six dimensions and finds distinct collapse patterns, including difficulty realizing requested temporal variation faithfully.
Problem
Global diversity scores obscure where variation collapses, while repeated outputs from the same prompt limit creative exploration.
Source: §1 · p. 1Method
DiVid diagnoses Semantic, Style, Subject, Scene, Motion, and Camera diversity with dimension-specific pipelines and joint quality and faithfulness analysis.
Source: §1 · p. 2Results
Takeaways & Limitations
The findings point to dimension-aware training objectives and control signals rather than a single universal diversity metric.
Source: §1 · p. 2MPD metrics across dimensions use different representation spaces and are not directly comparable.
Source: §5.2 · p. 7
Abstract
from arXiv · showhide
Despite remarkable progress, video generation models often produce highly similar outputs when repeatedly sampled from the same prompt, limiting their usefulness for creative exploration. Existing diversity evaluations primarily rely on global scalar metrics, which obscure where diversity collapses in the spatiotemporal space of videos. We introduce DiVid, a dimension-level diagnostic framework that decomposes video generation diversity into six interpretable dimensions: Semantic, Style, Subject, Scene, Motion, and Camera. Each dimension is measured through a reproducible computer-vision pipeline and analyzed alongside quality and instruction faithfulness to examine potential trade-offs. Systematic evaluation of representative video generation models reveals that diversity is highly dimension-specific: models with strong global diversity scores still collapse on specific factors, particularly Motion and Camera. These rankings persist after filtering unfaithful generations, indicating genuine capability differences rather than off-prompt outputs. Beyond measurement, controlled prompt interventions identify two fundamental bottlenecks: default mode convergence, where models fall back to dominant patterns under open-ended prompts; and realization gaps, where models fail to faithfully realize diverse, explicitly requested alternatives, particularly for temporal factors. The larger faithfulness losses for temporal factors highlight the difficulty of controlling motion and camera variation through text alone. DiVid thus shifts the study of diversity from measuring whether it exists to diagnosing where and why it collapses, and provides actionable directions for dimension-aware training objectives and control signals. The framework will be released to facilitate future research on diverse and controllable video generation.
1. Introduction
DiVid addresses the limits of global diversity scores by diagnosing variation across six dimensions and evaluating diversity alongside quality and faithfulness. Its evaluations reveal dimension-specific model failures and two bottlenecks in realizing diverse outputs.
Motivation
- Limited diversity can require excessive resampling, and locating the collapsed dimension is necessary for targeted recovery.The introduction also connects homogenization to possible training-data or objective biases.Source: §1 · p. 1
- Existing global metrics can give similar scores to models that vary different factors, obscuring whether collapse concerns scenes, camera motion, subject appearance, or style.Figure 1 illustrates that DiVid assigns the expected ordering across dimensions where global metrics such as TCE and TIE can miss factor-level differences.Source: §1 · pp. 1–2, Figure 1 · p. 2
Approach
- DiVid profiles Semantic, Style, Subject, Scene, Motion, and Camera diversity, using dimension-specific computer-vision pipelines alongside quality and faithfulness metrics.The framework distinguishes holistic Semantic and Style dimensions from factor-level dimensions.Source: §1 · p. 2
Results
- No model dominates every diversity dimension; models with strong global scores still collapse on Motion or Camera, and these rankings persist after unfaithful generations are filtered.The filtering result indicates genuine capability differences rather than off-prompt variation.Source: §1 · p. 2
- Open-ended prompts lead models toward dominant patterns, while explicitly requested alternatives are not consistently realized, particularly for Motion and Camera.The paper names these bottlenecks default-mode convergence and realization gaps.Source: §1 · p. 2
2. Related Work
Prior work evaluates video quality and faithfulness, global diversity, and prompt-aware variation, but fine-grained attribution and systematic video diversity evaluation remain limited.
Video Generation Evaluation
- Video benchmarks emphasize quality, alignment, and temporal consistency [26, 41, 53], while diversity metrics often use holistic global scores [16] [26] [27] [69].The passage also notes image-generation diversity evaluation [1] [3] [18] [60].Source: §2 · p. 3
- Prompt-aware methods separate model-induced from prompt-induced variation [28] [46], but provide limited attribution to subject, scene, or motion.Source: §2 · p. 3
Diversity and Homogeneity in Generative Models
3. DiVid: Framework Design
DiVid treats video diversity as a prompt-conditioned, set-level property and localizes variation across six dimensions with dimension-specific representations. Its reliability checks assess human agreement, pipeline validity, and whether factor-level diagnostics expose distinct failures.
3.1. Definition
- Video diversity is measured across independently sampled videos generated under the same prompt and settings, distinct from single-video quality or instruction faithfulness.A holistic embedding cannot localize where variation in a video's spatiotemporal signals originates.Source: §3.1 · p. 3
3.2. Six-Dimensional Diagnostic Pipeline
- The pipeline extracts dimension-specific representations, computes pairwise similarities, and aggregates set-level scores across holistic and factor-level dimensions.Semantic and Style use frame-level aggregation; Subject, Scene, Motion, and Camera pipelines combine detection, segmentation, tracking, and motion decomposition.Source: §3.2 · p. 3
3.3. Metric Reliability
- Human ratings significantly correlate with all six diversity metrics, and factor-level faithfulness achieves 85.4%–90.3% directional agreement.Diversity and faithfulness metrics have human-evaluation agreement above 0.667 Krippendorff’s α.Source: §3.3 · p. 4
- Validity checks find low invalid-record rates, 93.3% of inspected masks without salient omission or leakage, and unchanged dimension leaders under a stricter validity threshold.Prompt–dimension invalid rates range from 0.69% to 0.83%; the model-level rate ranges from 0% to 2.79%.Source: §3.3 · p. 4
- Factor-level diagnostics expose distinct failures: other factors leave 58.1–73.4% of held-out-factor variance unexplained, and selective retrieval succeeds in 74.7–93.3% of prompts.Removing one factor changes the prompt-level top-model set for 19.5–26.7% of prompts.Source: §3.3 · p. 4
3.4. Additional Metrics
- Instruction faithfulness scores whether generated videos satisfy prompt constraints for applicable factor dimensions, while quality metrics assess visual fidelity and motion quality.TCE and TIE provide global set-level diversity references using CLIP and Inception feature spaces, respectively.Source: §3.4 · pp. 4–5
- Table 1 reports faithfulness, quality, and diversity profiles macro-averaged over available prompt-level measurements for 206 general prompts.It separates open-source models, closed-source models, and two ablations.Source: Table 1 · p. 5
4. Main Evaluation
Across the evaluated models, diversity rankings reverse across dimensions, so global scores can conceal specific collapses. Joint evaluation also finds trade-offs with faithfulness and quality, while model preference depends on the diversity weight.
4.1. Experimental Setup
- The benchmark aggregates 206 prompts from VBench-Category, VBench-Dimension, and T2V-CompBench; each model generates 10 five-second, 16:9 videos per prompt.Open-source models use guidance scale 5.0, with Wan2.2-5B also evaluated at 3.0 and 7.0.Source: §4.1 · p. 5
4.2. Dimension-Specific Diversity
- Figure 3 shows dimension-specific rank reversals: Wan2.2-5B leads the four content dimensions, while Hunyuan leads Motion and Camera.Six of seven models span at least three rank positions across the six dimensions.Source: Figure 3 · p. 5, §4.2 · pp. 5–6
4.3. Joint Analysis on Quality, Faithfulness, and Diversity
- Figure 4b’s six-dimension macro curve shifts the top model from Wan2.7 at wD ∈[0, 0.83] to Hunyuan at wD ≥0.84.The curve averages win rates, not raw MPD, and is a rough recommendation summary.Source: Figure 4 · p. 6, §4.3 · p. 7
4.2. Dimension-Specific Diversity
- Global TCE and TIE rank Wan2.2-5B first, but Hunyuan has 1.33× higher Motion MPD and 5.05× higher Camera MPD.Camera MPD spans a 10.8× gap, from 0.490 (Hunyuan) to 0.045 (CogVideo).Source: §4.2 · p. 6
4.3. Joint Analysis on Quality, Faithfulness, and Diversity
- HappyHorse leads instruction faithfulness (IF=0.986) and imaging quality (IQ=0.729), yet ranks last in four content diversity dimensions; 35 of 49 significant correlations are negative.The correlations are after FDR correction; Table 14 reports them.Source: §4.3 · p. 6
- After retaining three videos per model and prompt with faithfulness ≥0.8, dimension-level rankings remain unchanged; Wan2.2-5B retains more than 98.3% of its original MPD.Hunyuan still leads Motion and Camera on the jointly valid prompt sets.Source: §4.3 · p. 6, Figure 4 · p. 6
5. Locating Diversity Bottlenecks via Prompt Intervention
Controlled prompts vary one target dimension at a time to probe default convergence, cue responsiveness, and candidate realization. Results show that explicit alternatives can increase diversity, but temporal factors also expose faithfulness and execution bottlenecks.
5.1. Experimental Setup
- The study compares Omitted, Hinted, and Enumerated interventions against a fully specified paired baseline, changing only one target dimension at a time.Omitted removes the target description, Hinted supplies a generic cue, and Enumerated supplies explicit candidates; subject omission is undefined.Source: §5.1 · p. 7
- Intervention effects are paired MPD differences from Specified, averaged over 20 semantic groups; 7/7 agreement is robust and 6/7 is a strong trend.MPD metrics across dimensions use different representation spaces and are not directly comparable.Source: §5.1 · p. 7, §5.2 · p. 7
5.2. Default Mode Convergence
- Under Omitted prompts, Motion improves for only four models, with a mean change of +0.011; generic motion and camera cues yield more consistent gains.“In motion” and “performing a clear action” yield Motion gains of +0.069 and +0.054, respectively, with positive responses in 6/7 models; camera cues yield +0.210 and +0.213, with 7/7 agreement.Source: §5.2 · pp. 7–8
- Figure 5 illustrates default subject convergence: the open category “an animal” becomes “a golden retriever.”The passage describes this as one convergence case.Source: Figure 5 · p. 8
- For Scene, “somewhere” improves over Omitted by only +0.005, while explicit enumeration yields +0.313; temporal dimensions instead respond gradually to cues and enumeration.The authors suggest categorical Subject and Scene diversity is unlocked by explicit candidates, while temporal variation benefits from candidate planning plus trajectory and camera execution constraints.Source: §5.2 · p. 8
5.3. Realization Gap
- Figure 6 shows enumeration consistently raises diversity across all four factor dimensions for every model, while mean faithfulness falls relative to Hinted prompts, especially for Motion and Camera.The authors conclude that diversity and execution must be optimized jointly, and that text alone bottlenecks fine-grained motion control.Source: Figure 6 · p. 8, §5.3 · p. 8
- Models differ in realization faithfulness: Seedance, HappyHorse, and Wan2.7 show large Camera gains with high faithfulness, while CogVideo and Wan2.2-5B gain slightly with low faithfulness.The latter two also rank last on Camera in Table 1; HappyHorse faithfully realizes enumerated Subject and Scene candidates despite ranking last in their diversity.Source: §5.3 · p. 8
6. Conclusion
DiVid diagnoses video diversity across six dimensions and finds that global metrics can hide distinct collapse patterns. Its dimension estimates depend on automatic extraction, with motion and camera estimation from 2D frames identified as especially difficult.
6. Conclusion
- DiVid finds that global metrics can hide dimension-specific collapse, while CFG, open-ended prompts, and explicit alternatives do not uniformly control diversity, especially for Motion and Camera.The conclusion points toward dimension-aware training objectives and richer control signals beyond text prompts.Source: §6 · p. 8
Ethics Statement
- For evaluation, annotators had high education levels and were fairly compensated; generated content may occasionally contain biased but unharmful material.The benchmark uses publicly available text-to-video models and prompts from open-source benchmarks.Source: Ethics Statement · p. 8
A.1. Dimension Extraction
- The four factor dimensions depend on automatic extraction, and recovering motion and camera information from 2D frames remains the hardest part.The framework targets natural disentanglement and set-level relative rankings; it does not aim to improve extraction accuracy.Source: §A.1 · p. 13
A.2. Toward Dimension-Aware Diversity Enhancement
The appendix outlines dimension-aware enhancement directions and supporting framework details. It reports stable model ordering across generation-set sizes, describes the metric pipelines and aggregation, and states boundaries on cross-benchmark comparison.
A.2. Toward Dimension-Aware Diversity Enhancement
- Data, control, and inference should target diagnosed dimensions: candidate prompts can expand content diversity, but Motion and Camera remain harder to control, and scalar guidance does not select which factor to diversify.The authors suggest factor-specific objectives and structured temporal controls such as trajectories, keypoints, and camera paths; pilot inference adaptations did not improve dimensions consistently.Source: §A.2 · p. 13
B.1. Robustness of Generation Number
- With 10 versus 30 videos per prompt, six-dimensional model ordering remains the same on MPD and Vendi, while MPD stays stable and raw Vendi rises at a diminishing rate.Figure 7 reports the metric sensitivity comparison; Figure 8 visualizes intra- and inter-model homogenization using PCA.Source: §B.1 · p. 14, Figure 7 · p. 15, Figure 8 · p. 15
B.2. Additional Information about DiVid Framework
- A full six-dimension evaluation takes about 30 seconds per 10-video group on one A100 GPU, with encoders and geometric hyperparameters fixed across models.Each video is represented dimension-specifically, converted to a pairwise kernel, then aggregated within its prompt set.Source: §B.2 · p. 14
B.2.1. Details on Diversity Metrics
- TCE/TIE values computed from 10 videos per prompt support within-benchmark comparison only, not numerical comparison with prior 20-video results.The stated reason is sensitivity to sample count and retained eigenvalues.Source: §B.2.1 · p. 14
- Semantic diversity removes the prompt-alignment direction from CLIP video embeddings, while Style uses InceptionV3 features to capture global appearance.The semantic operation reduces shared prompt alignment but does not remove all semantic dependence on the prompt; style includes composition, texture, color, and lighting.Source: §B.2.1 · p. 14
- Subject and Scene metrics separate subject appearance from background variation, while Motion and Camera descriptors distinguish compensated subject movement from background camera movement.Motion and Camera use estimated trajectories and flow; only videos with valid trajectory and deformation estimates are included.Source: §B.2.1 · p. 14
B.2.2. Subject Annotation
- Subject annotations prioritize a clearly visible, recognizable foreground entity suitable for localization and segmentation; three annotators reviewed the initial labels.For 94.7% of prompts, all three annotators unanimously accepted the initial GPT-5 label; disagreements used majority vote.Source: §B.2.2 · p. 15
B.2.3. Set-Level Aggregation
- MPD is the expected distance between two uniformly sampled videos from a prompt–model set, and raw MPD values are compared only within the same dimension.Vendi Score provides an effective-mode perspective, while MPD is primary because its pairwise interpretation is direct.Source: §B.2.3 · p. 16
B.3. Validity Checks and Scope
Validity checks govern which subject and temporal measurements are retained, while sensitivity tests and scope boundaries clarify what the resulting metrics can and cannot represent.
Validity and statistics
- Subject tracking requires at least three valid masks, while temporal descriptors require minimum valid-point counts, confidence thresholds, and valid-pair coverage.A factor is excluded rather than scored zero when fewer than two videos or fewer than 30% of generated videos retain valid subject masks; Motion also requires both estimation branches to pass.Source: §B.3 · p. 16
- 96.60% of expected videos pass the complete mask-validity gate, and final prompt–dimension invalid rates stay below 0.83%.Across attempted temporal estimates, 96.70% of background-transform pairs, 94.21% of background-valid subject-trajectory pairs, and 98.48% of canonical-deformation pairs pass their gates; requiring at least 8 of 10 valid videos preserves each factor dimension’s leading model.Source: §B.3 · p. 16
- In 300 accepted mask overlays, human review passed 93.3%; remaining failures clustered in heavy occlusion and multi-object scenes.Reviewers checked salient omissions and leakage, complementing automated status flags.Source: §B.3 · p. 16
Sensitivity and scope
- Motion MPD rises from subtle breathing or twitches to direction or speed changes, and is highest when action type and amplitude change together.The sensitivity pilot varies motion descriptions for the same animal subject.Source: §B.3 · p. 16, Table 4 · p. 16
- Factor metrics require an explicit, localizable subject; Motion and Camera measure 2D image-plane variation, not physical correctness or full 3D camera pose.Prompts outside the subject scope are excluded; strong parallax, dynamic backgrounds, and shot cuts may require richer geometry, and Motion remains sensitive to tracking quality.Source: §B.3 · pp. 16–17
- DiVid uses dimension-matched representations, so MPD values are compared within—not across—dimensions, and alternative encoders can be substituted.The framework retains its factor definitions, pairwise aggregation, and evaluation protocol when encoders change.Source: §B.3 · p. 17
B.4. Human Evaluation
Human evaluation compares model pairs on dimension-specific diversity and factor-level faithfulness, with blinding and reliability checks. The resulting diversity scores align positively with human preferences.
Evaluation protocol
- Six trained annotators judged 150 randomly sampled prompt–model-pair items on six diversity dimensions and rated factor faithfulness on a 1–5 scale.Model identity, file names, left/right position, and video order were hidden or randomized; 240 valid judgments covered all items, with 90 judged by two annotators.Source: §B.4 · p. 17
Quality control
- Annotators received shared training, watched at least 80% of motion-related clips, and saw appearance and motion dimensions on separate screens.Gold items checked attention; all responses correctly identified the curated distinct-video sets as more diverse.Source: §B.4 · p. 17
Results
- All six diversity metrics correlate positively with human pairwise preferences: Spearman ρ = 0.531–0.604 and Kendall τb = 0.471–0.539, with p < 0.001.Factor-faithfulness direction agreement is 85.4%–90.3%, and Krippendorff’s α exceeds 0.667; the authors support model-level ranking and diagnostic use.Source: §B.4 · p. 17, Table 5 · p. 17, Table 6 · p. 17
B.5. Utility of Factor Level Metrics
The appendix adds statistical tests, reference comparisons, and sensitivity analyses to the factor-level utility results. Across these analyses, dimension-specific rankings and diagnostics remain informative under several evaluation settings.
B.5. Utility of Factor Level Metrics
- The other three factors leave 58.1–73.4% of held-out factor variance unexplained; removing one changes the prompt-level top-model set for 19.5–26.7% of prompts.Each factor retrieves selective video pairs in at least 74.7% of prompts, supporting non-interchangeable diagnostic signals.Source: §B.5 · p. 18, Table 7 · p. 19, Table 8 · p. 19
C.1. Complete Model-Level Statistics
- Wan2.2-5B leads Hunyuan by 0.027–0.054 MPD across four content dimensions, while Hunyuan leads by 0.107 in Motion and 0.394 in Camera.The prompt-paired confidence intervals exclude zero; both temporal differences have q < 5.0 × 10^-20, substantiating the cross-dimensional reversal.Source: §C.1 · p. 18
C.2. Real-Video Retrieval Diversity Reference
- Retrieved real videos rank first in Semantic, Style, Subject, Scene, and six-dimension average diversity, but fourth in Motion and Camera.Their quality scores are comparable to evaluated generators; the result shows broad content variation need not imply equally rich temporal variation.Source: §C.2 · p. 18
C.3. Quality–Faithfulness–Diversity Misalignment
- Across seven model means, IF–Subject has the strongest negative association (ρ = −0.786), while IF correlates positively with Motion and Camera.At prompt level, 49 of 126 tested correlations remain significant after FDR correction: 35 negative and 14 positive.Source: §C.3 · p. 18
C.4.1. Threshold-Gated Diversity
- Among faithfulness-filtered models, Wan2.2-5B leads Style, Subject, and Scene, while Hunyuan leads Motion and Camera across τ ∈ [0.50, 1.00].Semantic switches to Seedance at τ = 0.85, but the 0.0015–0.0021 margin and 51.8%–55.2% bootstrap top-1 frequency make it a near tie, not a robust reversal.Source: §C.4.1 · p. 18
C.4.2. Continuous Preference Weights
- Wan2.7 ranks first for diversity preference weights wD ∈ [0, 0.83], and Hunyuan for wD ∈ [0.84, 1], with broad top-two coverage.The recommendation remains dimension dependent; the separate quality-oriented scenario weights AQ, IQ, and diversity at 0.25/0.25/0.50 when instruction fidelity is not the explicit constraint.Source: §C.4.2 · pp. 18–19, Figure 9 · p. 19
C.5. Cross-Prompt-Set Transfer
- Across six diversity dimensions, 83.3% of 21 model-pair orders are preserved on average on VBench and T2V-CompBench.Style and Camera preserve every pairwise order; Subject, Scene, and Motion show limited top-model changes under compositional prompts.Source: §C.5 · p. 19
C.6. Cross-Model Complementarity
This block reports cross-model complementarity, dimension-specific examples, and supplementary prompt-construction and statistical details. The findings distinguish a model’s standalone diversity from the diversity added by pairing it with another model.
C.6. Cross-Model Complementarity
- The union of two models improves over the matched within-model baseline on 97.0% of prompt–dimension comparisons on average.CogVideo and HappyHorse, despite relatively low standalone content diversity, form the most complementary pair in four dimensions; routing should consider whether default output preferences differ.Source: §C.6 · pp. 19–20
C.7. Dimension-Specific Case Studies
- The case studies distinguish repeated concept realization from style collapse, repeated foreground identity from scene collapse, and repeated trajectories from repeated camera policy.Figures 13–15 show these holistic, composition, and temporal collapse patterns.Source: §C.7 · p. 20
D. Supplementary Materials for Section 5
- The supplementary materials cover controlled-prompt construction, statistical protocol, faithfulness diagnostics, and representative failure modes.This appendix supplements Section 5.Source: D. Supplementary Materials for Section 5 · p. 20
D.1. Complete Experimental Design and Prompt Construction
- The controlled design changes one target dimension at a time from a fully specified paired baseline; settings are Specified, Omitted, Hinted, and Enumerated.Subject omission is undefined because a valid prompt must include a subject; the set contains 20 reviewer-screened prompts spanning people, animals, vehicles, and varied environments.Source: §D.1 · p. 20
D.2. Statistical Details
- Model means average paired MPD differences over 20 prompt groups, while panel-level tests use the 20 group means as statistical units.Reported analyses include bootstrap confidence intervals, block win rate, paired Hedges gz, Wilcoxon p values, and BH-adjusted q values.Source: §D.2 · p. 21
D.3. Faithfulness Evaluation and Realization Cost
The section compares hinted prompts with explicit enumeration to measure the realization cost of stronger control, then uses controlled prompts to expose default convergence and incomplete realization.
D.3. Faithfulness Evaluation and Realization Cost
- The authors compare Enumerated with Hinted within each model, semantic group, and target dimension because Hinted prompts are less specific and easier to satisfy, measuring the realization cost of explicit control.Source: §D.3 · p. 21, Table 23 · p. 37
D.4. Failure Modes Revealed by Controlled Prompting
- 0.136, 0.147, and 0.871 are HappyHorse's Subject MPD values for the animal prompt under Specified, Hinted, and Enumerated settings; broad cues still yield mostly the same dog.Source: §D.4 · p. 21
- Motion MPD for Hunyuan's road-person prompt is 0.464 (Specified), 0.259 (Hinted), 0.325 (Omitted), and 0.613 (Enumerated); Camera MPD is 0.225, 0.125, 0.216, and 0.752, respectively.The Camera values follow the same setting order: Specified, Hinted, Omitted, Enumerated.Source: §D.4 · p. 21, Figure 17 · p. 28
- Enumerated Hunyuan Motion candidates score F = 5, 5, 4, 5, and 3, while corresponding Camera candidates include an F = 2 despite a large MPD gain.Hinted Motion videos all receive F = 5; the cases motivate reporting diversity and faithfulness separately.Source: §D.4 · p. 21, Figure 18 · p. 28