Source-linked AI summary
ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraints
Meiqi Wu, Jiashu Zhu, Xiaokun Feng, Chubin Chen, Chen Zhu, Bingze Song, Fangyuan Mao, Jiahong Wu, Xiangxiang Chu, Kaiqi Huang
TL;DR
Video generators struggle with imaginative prompts containing rare, long-distance semantic relationships, while existing test-time scaling uses fixed search spaces and rewards. ImagerySearch adapts both to prompt semantics and is evaluated alongside a new benchmark, LDT-Bench. It achieves state-of-the-art results on VBench and LDT-Bench, with especially strong gains on LDT-Bench.
Problem
Video generators degrade on imaginative prompts with long-distance semantic relationships, and existing test-time scaling methods use fixed search spaces and static rewards.
Source: §1 · p. 2Method
ImagerySearch adapts its inference search space and reward to prompt semantics, while LDT-Bench evaluates long-distance semantic prompts.
Source: §1 · pp. 2–3Results
ImagerySearch achieves state-of-the-art results on both VBench and LDT-Bench, with especially strong gains on LDT-Bench.
Source: §6 · p. 10Takeaways & Limitations
The reported results demonstrate effectiveness for text-to-video generation under long-range semantic conditions.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Video generation models have achieved remarkable progress, particularly excelling in realistic scenarios; however, their performance degrades notably in imaginative scenarios. These prompts often involve rarely co-occurring concepts with long-distance semantic relationships, falling outside training distributions. Existing methods typically apply test-time scaling for improving video quality, but their fixed search spaces and static reward designs limit adaptability to imaginative scenarios. To fill this gap, we propose ImagerySearch, a prompt-guided adaptive test-time search strategy that dynamically adjusts both the inference search space and reward function according to semantic relationships in the prompt. This enables more coherent and visually plausible videos in challenging imaginative settings. To evaluate progress in this direction, we introduce LDT-Bench, the first dedicated benchmark for long-distance semantic prompts, consisting of 2,839 diverse concept pairs and an automated protocol for assessing creative generation capabilities. Extensive experiments show that ImagerySearch consistently outperforms strong video generation baselines and existing test-time scaling approaches on LDT-Bench, and achieves competitive improvements on VBench, demonstrating its effectiveness across diverse prompt types. We will release LDT-Bench and code to facilitate future research on imaginative video generation.
1 Introduction
ImagerySearch adapts test-time search to imaginative prompts, where long-distance semantic relationships and scarce training examples challenge video generation. The paper also introduces LDT-Bench, and reports improvements on LDT-Bench and VBench.
1 Introduction
- Imaginative scenarios challenge video generators because long-distance semantic relationships fall beyond training distributions and imaginative combinations are scarce in mainstream datasets.Existing test-time scaling methods use predefined sampling spaces and static rewards, limiting their adaptability to open-ended creative generation.Source: §1 · p. 2
- ImagerySearch adapts sampling granularity to prompt semantic span and uses an adaptive reward to encourage semantic alignment.Its design is motivated by the idea that people iteratively refine visual imagery in response to language.Source: §1 · pp. 2–3
- LDT-Bench contains 2,839 challenging concept pairs and uses ImageryQA to assess element coverage, semantic alignment, and anomaly detection.The benchmark is designed specifically for long-distance semantic prompts.Source: §1 · pp. 2–3
- ImagerySearch consistently improves video quality and semantic alignment on LDT-Bench and VBench for long-distance semantic prompts.Experiments also report that general video models and existing test-time scaling approaches degrade in quality and alignment on these prompts.Source: §1 · pp. 2–3
2 Related Work
Related work covers video-generation models, test-time scaling strategies, and video evaluation benchmarks. It identifies long-distance semantic understanding as an under-addressed evaluation target.
2 Related Work
- Large text-to-video models can generate coherent videos and generalize to complex scenarios, but long-range semantic video data remain scarce and costly to collect.Test-time scaling methods used in ImagerySearch require no additional training.Source: §2 · p. 3
- Test-time scaling selects better outputs with rewards, using strategies such as Best-of-N, particle sampling, and beam search.Current approaches optimize search and reward strategies for general text-to-video generation rather than specifically targeting long-distance semantic prompts.Source: §2 · p. 3
- Video evaluation has expanded to multidimensional benchmarks such as VBench, while existing benchmarks still struggle to assess long-distance semantic understanding.Other work has refined evaluation along dimensions such as frame realism and temporal coherence.Source: §2 · p. 3
3 ImagerySearch
ImagerySearch frames video generation as a search over noise candidates, with rewards evaluating video quality. It adapts search and reward design to prompt semantics rather than relying on fixed search spaces and static rewards.
3.1 Preliminaries
- ImagerySearch formulates generation as searching for noise inputs, with reward functions assessing video quality and search algorithms exploring and selecting candidates.Unlike prior test-time scaling approaches with fixed noise spaces and static rewards, it supports flexible rewards and adaptive noise selection.Source: §3 · p. 4, §3.1 · p. 4
3.2 Dynamic Search Space
- SaDSS adjusts the number of candidates according to the average embedding distance between key prompt entities, using a base count and a sensitivity factor.It shrinks the search space for short-distance prompts and enlarges it for long-distance prompts.Source: §3.2 · pp. 4–5
- Tailoring search scope to prompt difficulty encourages more diverse visual hypotheses for challenging prompts without unnecessary computation on simple prompts.This design is motivated by the idea that semantically distant concepts require more effort to construct as mental imagery.Source: §3.2 · pp. 4–5
3.3 Adaptive Imagery Reward
- ImagerySearch evaluates candidates at noise levels S = {5, 10, 20, 45} by completing denoising from a partially denoised latent and scoring the resulting video.The selected levels form the Imagery Schedule; adjacent denoising steps are observed to change the latent only marginally.Source: §3.3 · p. 5
- AIR adapts candidate evaluation to prompt semantic difficulty by combining VideoAlign components MQ, TA, and VQ with an extensible reward, scaled by semantic distance.The extensible term can use rewards such as VideoScore, VMBench, UnifiedReward, or VisionReward.Source: §3.3 · p. 5
4 LDT-Bench
LDT-Bench addresses the lack of a unified benchmark for imaginative prompts whose objects and actions are rarely co-occurring. It builds long-distance prompts from object–action and action–action pairs and evaluates generated videos across three dimensions.
4 LDT-Bench
- LDT-Bench provides a dedicated benchmark for complex prompts with long semantic distances, where entities such as objects and actions rarely co-occur.The paper positions it as a systematic alternative to prior qualitative analyses on a few cases.Source: §4 · p. 5
4.1 Prompt Suite
- Prompts pair objects with uncommon actions or combine semantically distant or contradictory actions, drawing on 1,938 objects and 901 actions.T5 embeddings projected by PCA into 2D are used to measure pairwise Euclidean distance; the 160 most distant pairs from each candidate set form 320 prompts.Source: §4.1 · pp. 5–6
- GPT-4o turns selected element pairs into fluent prompts, which DeepSeekR1 and human annotators jointly check before inclusion.Figure 3 also contrasts LDT-Bench’s category coverage and semantic-distance distribution with other benchmarks.Source: §4.1 · p. 6, Figure 3 · p. 6
4.2 Imagery Evaluation Metrics
- ImageryQA evaluates generated videos for element coverage, visual quality and aesthetics, and anomalies using three MLLM-based tasks.ElementQA asks about object presence and action performance; AlignQA assesses visual quality and aesthetics, while AnomalyQA identifies anomalous outputs.Source: §4.2 · pp. 6–7
5 Experiments
Experiments compare Wan2.1-based ImagerySearch with general video generators and test-time scaling methods on LDT-Bench and VBench. The reported results show gains on LDT-Bench, strong VBench performance, robustness across semantic distances, and benefits from the adaptive search components.
5.2 Comparison with Other Generation Models
- 8.83% improvement over the Wan2.1 base model on LDT-Bench, where ImagerySearch also achieves the best average performance.The paper reports consistently superior performance to other test-time scaling approaches.Source: §5.2 · p. 7, Table 1 · p. 7
- On LDT-Bench prompts, ImagerySearch achieves the best VBench overall score and ranks highest on Dynamic Degree and Subject Consistency.The reported examples show it reproducing specified subjects and their associated actions.Source: §5.2 · pp. 8–9, Table 2 · p. 8
- Across increasing semantic distances, ImagerySearch maintains nearly constant VBench scores while competing methods fluctuate more.The figure also compares scaling behavior across MQ, TA, VQ, and Overall VideoAlign scores as inference-time computation increases.Source: Figure 5 · p. 8, §5.2 · p. 9
5.3 Test-time Scaling Law Analysis
- On Wan2.1, ImagerySearch improves monotonically on VideoAlign’s MQ, TA, and VQ metrics as NFEs increase, while baselines plateau around 1 × 10^3 NFEs.The method’s gains continue beyond the approximate plateau corresponding to the 30th timestep; its advantage is more pronounced on overall VideoAlign.Source: Figure 5 · p. 8, §5.3 · p. 9
5.4 Ablation Study
- Adding SaDSS or AIR individually surpasses the baseline, and combining them yields the best performance, supporting their complementary roles.The ablation table identifies the full ImagerySearch configuration as the best-performing configuration.Source: §5.4 · p. 9, Table 3 · p. 9
- Dynamic search space reaches 83.48% VBench, compared with 81.22% for the reported static-weight configurations.The paper reports gradual improvement for static weights of 0.5 and 0.9, while the dynamic approach scores higher.Source: §5.4 · p. 9, Table 3 · p. 9
- The proposed search strategy outperforms the tested alternatives, including BON and Particle Sampling.The comparison is reported in the Search rows of the ablation table.Source: §5.4 · p. 9, Table 3 · p. 9
- As reward weights vary from 0.2 to 1.2, TA improves while MQ and VQ remain relatively stable; the dynamic-reward approach remains superior.The comparison covers VBench scores across different models.Source: Figure 5 · p. 8, §5.4 · p. 9
6 Conclusion
ImagerySearch is presented as an adaptive test-time method for video generation under long-distance semantic conditions, and LDT-Bench as a benchmark for evaluating such prompts. The conclusion reports state-of-the-art results on both benchmarks, especially strong gains on LDT-Bench; the implementation invokes the search at selected noise levels.
6 Conclusion
- ImagerySearch attains state-of-the-art results on VBench and LDT-Bench, with especially strong gains on LDT-Bench.The authors identify more flexible reward mechanisms as future work.Source: §6 · p. 10
A The Selection of Imagery Schedule
- ImagerySearch is invoked at noise levels S = {5, 20, 30, 45}, termed the Imagery Schedule.The schedule selects a limited set of steps because adjacent denoising steps modify the latent video only marginally, while substantial deviations emerge at pivotal steps.Source: A The Selection of Imagery Schedule · p. 16
B More Details About Imagery Evaluation Metrics
The text encoder supports both semantic-distance estimates and reward alignment, and T5 can be replaced by CLIP without changing downstream components.
B.1 More Text Encoders
- CLIP can replace T5 as a drop-in encoder, supplying semantic distances for adaptive search and similarities for the reward’s alignment term.The passage states this replacement preserves the adaptive mechanisms’ effectiveness without changing downstream components.Source: §B.1 · p. 16
B.2 More Analysis about Prompt Suite.
LDT-Bench spans diverse object and action categories, combining common classes with rarer ones. Its prompt suite emphasizes long-distance semantic relationships that prior benchmarks largely miss.
B.2 More Analysis about Prompt Suite
- Sports & Wellness and Daily Services each contribute 300 action prompts, while Animal and Artifact together exceed half of object samples.The object treemap also retains room for rarer classes.Source: §B.2 · p. 16
- Frequent nouns include cricket, person, and remote, while action verbs such as play, join, use, and handle span varied semantic domains.The object word cloud follows stop-word filtering.Source: §B.2 · p. 16
- LDT-Bench balances head and tail categories, covering a richer mix of objects and actions than existing datasets while emphasizing difficult long-distance relationships.Animal and Artifact together exceed half of object samples, while rarer classes remain represented.Source: §B.2 · p. 16
B.3 ImageryQA Implementation Details.
ImageryQA evaluates prompt coverage, semantic alignment, and anomalies through distinct question types and model-based answers. Its implementation uses repeated responses for uncertain questions and concentrates search on key denoising steps.
B.3 ImageryQA Implementation Details
- ElementQA checks prompt objects and actions, AlignQA assesses first-frame image quality and aesthetics, and AnomalyQA evaluates abnormal events.For the example prompt, ElementQA asks about the person, polishing furniture, and packing cleaning products.Source: §B.3 · p. 16
- Attention focus regions remain nearly identical across adjacent denoising steps, so analysis and search concentrate on the few steps with pronounced changes.The attention heatmaps project the 13th layer onto the first video frame.Source: Figure S1 · p. 17
- The prompt-suite overview visualizes action super-categories, object super-categories, and frequent object–action and action–action prompts using charts and word clouds.The action categories use a horizontal bar chart and the object categories a treemap scaled by class count.Source: Figure S2 · p. 17
- ElementQA uses five Qwen2.5-VL-72B-Instruct answers per question, AnomalyQA collects three GPT-4o responses, and AlignQA uses one Q-Align score.The passage motivates repeated answers for questions with inherent uncertainty and notes Q-Align directly outputs a quantitative aesthetic score.Source: §B.3 · p. 17
C Experimental Setup–Model details
The setup specifies Wan2.1-1.3B and fixed imagery schedules, with adaptive adjustment of the TA coefficient. Additional examples report coherent outputs across LDT-Bench and VBench, including higher ImageryQA scores than Wan2.1 on two measures.
C Experimental Setup–Model details
- The implementation uses Wan2.1-1.3B, imagery schedule {5, 20, 30, 45}, and imagery size schedule {10, 5, 5, 5, 5}, with β = γ = 1.0 and dynamically adjusted α.The parameter choice follows the reported same selection trend for VQ and MQ and opposite trend for TA.Source: C Experimental Setup–Model details · p. 17
- On the same prompt, ImagerySearch scores higher than Wan2.1 on Q4 aesthetic quality and Q5 visual anomalies.The example depicts polishing furniture and packing cleaning products; Wan2.1 fails to depict the person and described actions.Source: Figure S3 · p. 18
D More Examples
- ImagerySearch examples show vivid, coherent videos on LDT-Bench and VBench, including action–action and object–action prompts.The LDT-Bench examples cover five action–action rows and three object–action rows; the VBench examples show additional action–action cases.Source: D More Examples · p. 18
E Error Analysis
ImagerySearch has the highest mean and tightest spread in the VBench long-distance prompt analysis. The appendix also describes reward-weight trends and uniform frame sampling used to present additional examples.
E Error Analysis
- ImagerySearch has a higher mean score and tighter interquartile range than the compared methods, while Evosearch has a comparable median but greater dispersion.Wan2.1 and Video-T1 show lower central scores and wider quartile spans; the figure marks each model’s mean with a white diamond and overlays individual scores.Source: E Error Analysis · p. 18, Figure S8 · p. 23
- MQ and VQ follow nearly identical selection trends while TA moves oppositely, motivating fixed MQ and VQ coefficients and prompt-adaptive TA weighting.The figure contrasts an action–action example with an object–action example under different weight settings.Source: Figure S4 · p. 19
- The additional LDT-Bench and VBench examples display 16 uniformly sampled frames from each 33-frame video.The captions identify the sampled frames as the images shown below each prompt.Source: Figure S5 · p. 20, Figure S6 · p. 21, Figure S7 · p. 22