PaperlayerView exact revision on arXiv

Source-linked AI summary

Golden Anchor for Video Generation

Jinyuan Xu, Shitong Shao, Lichen Bai, Zikai Zhou, Guoxi Huang, Nantheera Anantrasirichai, Zeke Xie

arXiv:2610.13217v1cs.CVcs.LG

TL;DR

Image quality scores do not reliably indicate whether an anchor will yield a high-quality video, raising a mismatch for T2I-to-I2V pipelines. GAL trains the T2I generator using preferences derived from videos rendered by a frozen I2V model. Across four I2V models, it improves video and motion metrics, including when evaluated on models not used to construct preferences.

  • Problem

    Image aesthetics and quality are nearly uncorrelated with downstream video rewards, while structure-and-texture scores are negatively correlated with visual and motion quality.

    Source: §1 · pp. 1–2

  • Method

    GAL renders candidate anchors with a frozen I2V model, converts video rewards into image-level preferences, and fine-tunes the T2I generator with offline DPO.

    Source: §1 · p. 3

  • Results

    GAL raises the VBench average by 2.97%–8.46% over the untuned generator across all four I2V models, with consistent dynamic-degree gains.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Video-derived preferences can improve anchors for downstream video generation, with gains transferring to three I2V models not used to construct preferences.

    Source: §1 · p. 3

    Preference construction uses only HunyuanVideo-1.5 renders, and GAL depends strongly on the choice and reliability of the video-reward model.

    Source: §6.6 · pp. 9–10

Abstract

from arXiv · show

Video generation supports digital content creation, with applications in virtual avatars, AI-driven filmmaking, interactive game engines, and world models. State-of-the-art systems achieve controllability primarily through image-to-video (I2V) generation, in which an injected anchor image determines the identity, composition, and style of the resulting video. In modern pipelines, this anchor is synthesized by a text-to-image (T2I) generator. This raises a largely overlooked question: Are images obtained directly from the T2I model truly suitable for current video models? Our analysis shows that anchor aesthetic and quality scores are weakly correlated with downstream video quality. The structure-and-texture score is negatively correlated with I2V video quality in both the visual and motion dimensions. Fine-tuning the T2I model on an image reward accordingly yields anchors with visibly denser structural and character detail: they score higher on image metrics, yet lower the video reward on every I2V model we test and change downstream motion unpredictably. We therefore introduce Golden Anchor Learning (GAL), to our knowledge the first framework for learning golden anchors by optimizing a T2I generator with downstream video feedback. GAL evaluates candidate anchors through the videos they induce under a frozen I2V model and converts video rewards into image-level preferences for DPO fine-tuning of the T2I generator. On all four I2V models, GAL raises the VBench average by $2.97\%$--$8.46\%$ over the untuned generator. The rendered clips also score higher under VideoReward-Overall, and their extracted frames under HPSv3 and UniPercept. GAL yields consistent motion gains across all four I2V models, as measured by a $5.62\%$--$38.24\%$ increase in VBench dynamic degree. Although GAL is trained only on HunyuanVideo-1.5 renders, these gains extend to the other three I2V models.

1 INTRODUCTION

Image-side scores do not reliably predict downstream video quality, motivating anchors optimized for the videos they induce. Golden Anchor Learning (GAL) uses video-derived preferences to fine-tune the T2I generator, improving video and motion metrics across four I2V models.

Motion evidence

  • In the shown examples, GAL anchors produce the highest per-interval in-scene motion across all four I2V models.Figure 1 compares clips generated with the same prompt and renderer from Base, GAL-Image, and GAL anchors.Source: Figure 1 · p. 2

Finding

  • Across 49,416 records, anchor aesthetics and quality are nearly uncorrelated with video rewards, while structure and texture are negatively correlated with visual and motion quality.Image-based preference tuning produces sharper anchors but lowers VideoReward-Overall on all four evaluated I2V models, with motion changing inconsistently.Source: §1 · p. 2, Figure 2 · p. 2

Contributions

  • GAL is the first framework described here to learn anchors using downstream video feedback rather than selecting anchors by image-side criteria.The paper formalizes anchor generation as a learning problem.Source: §1 · p. 3

Method

  • GAL converts downstream video rewards into image-level preferences for offline DPO fine-tuning of the T2I generator, without additional rendering or scoring at inference.Candidate anchors are evaluated through videos rendered by a frozen I2V model.Source: §1 · p. 3

Contributions

  • GAL raises the VBench average by 2.97%–8.46% over the untuned generator across all four I2V models.Rendered clips also score higher under VideoReward-Overall, with extracted frames scoring higher under HPSv3 and UniPercept; dynamic degree rises by 5.62%–38.24%.Source: §1 · p. 3

2 RELATED WORK

Prior work addresses image-anchored video generation through image conditioning, rendering, inference-time feedback, or preference learning. GAL instead optimizes the upstream anchor generator using preferences derived from rendered videos.

Image-anchored video generation pipelines

  • Prior methods modify image conditioning, rendering dynamics, or inference-time feedback; GAL instead shifts the upstream anchor distribution through training.The paper presents this approach as complementary to those methods.Source: §2 · p. 3

Preference learning across image and video

  • Image-domain preference rewards can improve anchor appearance, but their effects on downstream video motion are unstable across I2V models.Video-domain rewards score rendered clips; GAL uses them to optimize the anchor image while keeping the I2V model frozen.Source: §2 · p. 3

3 PROBLEM SETUP: THE GOLDEN ANCHOR OF AN I2V RENDER

The paper defines an anchor’s value through the quality of the video it induces under a fixed, frozen I2V model and video reward. This downstream objective motivates offline preference optimization.

3 PROBLEM SETUP: THE GOLDEN ANCHOR OF AN I2V RENDER

  • An anchor is not the final output: its value is assessed through the video it induces under the frozen downstream I2V model.The pipeline samples an anchor from the upstream T2I generator, then renders a clip conditioned on it.Source: §3 · p. 4
  • A golden anchor has high downstream value when it tends to induce highly rewarded videos across rendering noise for a fixed prompt, I2V model, and reward.The T2I distribution is optimized toward such anchors.Source: §3 · p. 4
  • Because estimating anchor value requires rendering and scoring videos, GAL builds a fixed preference dataset offline and optimizes the T2I generator with DPO.This keeps the expensive I2V renderer out of the training loop.Source: §3 · p. 4

4 GAL: GOLDEN ANCHOR LEARNING

GAL constructs image preferences from rendered-video rewards, then fine-tunes the upstream T2I generator with DPO while keeping the I2V renderer and reward model frozen. Its implementation uses a task LoRA within a differential-LoRA scaffold.

4 GAL: GOLDEN ANCHOR LEARNING

  • GAL first builds a fixed preference dataset from video rewards, then optimizes the upstream image generator while keeping the I2V renderer and video reward frozen.Video rendering, scoring, and preference construction are performed offline.Source: §4 · p. 4, Figure 3 · p. 5

4.1 CONSTRUCTING CROSS-MODAL PREFERENCES

  • For each prompt, GAL samples four sibling anchors, renders each once with a frozen I2V model, and scores the resulting clips with a frozen video-reward model.The scalar Overall score is used for training; per-axis scores are retained for analysis, and near-ties are removed to reduce sensitivity to I2V sampling noise.Source: §4.1 · p. 4

4.2 PREFERENCE OPTIMIZATION VIA DPO

  • Cross-modal DPO trains on image pairs whose preference labels come from rendered-video scores, using shared noising context for winner, loser, trainable model, and reference model.The renderer and reward model are used only to build the preference dataset.Source: Figure 3 · p. 5, §4.2 · p. 5

4.3 FEW-STEP IMPLEMENTATION

  • The few-step Z-Image-Turbo implementation trains a task LoRA within a differential-LoRA scaffold; removing the scaffold reverses the motion gain despite image-side improvements.The frozen preset adapter is removed at inference, and the training objective targets the deployed generator.Source: §4.3 · p. 6

5 ANALYSIS OF GOLDEN ANCHOR LEARNING

The analysis explains why image and video objectives can diverge and how GAL uses video-derived preferences rather than image gradients. It also motivates margin filtering as a practical response to noisy single-render comparisons.

5 ANALYSIS OF GOLDEN ANCHOR LEARNING

  • GAL uses video-derived labels and margin filtering at the objective level, with supporting derivations in Appendix A and trained-system evaluation in Section 6.3.Source: §5 · p. 6
  • Image-improving directions need not align with downstream video value because the frozen renderer’s response to an anchor also affects that value.GAL does not compute either gradient; downstream value enters training through preference labels estimated by rendering and scoring candidate anchors.Source: §5 · p. 6
  • The margin removes near-tie comparisons vulnerable to one-render sampling noise, producing a smaller, more separated preference set as the margin increases.The multi-seed evaluation tests whether the learned effect persists under downstream rendering stochasticity; the margin is not treated as proof that one render exactly estimates O(x, p).Source: §5 · p. 7

6 EXPERIMENTS

The experiments compare rendered-video feedback with image-based feedback across four frozen I2V models, then examine reward-signal and training-scaffold choices. GAL improves video and motion metrics across models, while the discussion notes limits tied to reward supervision and the single training backend.

6.1 SETTINGS

  • HunyuanVideo-1.5 alone supplies preference labels; open-source evaluations average three I2V seeds, while Kling v3 Pro uses one seed on 50 held-out prompts.Training, VBench, and Kling prompts are pairwise disjoint.Source: §6.1 · p. 7

6.2 QUALITATIVE COMPARISON

  • In the shown examples, GAL-VideoReward has the highest per-interval in-scene motion in all 20 one-second intervals across the four I2V models.The examples compare Base, GAL-Image, and GAL-VideoReward under the same prompt and renderer.Source: §6.2 · p. 7

6.3 MAIN COMPARISON ACROSS I2V MODELS

  • GAL raises VBench average by 2.97%–8.46% over Base across four I2V models, with dynamic degree gains of 5.62%–38.24% and higher reward scores.Only HunyuanVideo-1.5 supplies preference labels; gains transfer to LTX-Video, CogVideoX-1.5, and Kling v3 Pro.Source: §6.3 · p. 7, Table 1 · p. 8, Table 2 · p. 8

6.4 REWARD-SIGNAL ABLATION WITHIN GAL

  • Among reward variants, GAL-VideoReward has the highest VBench average on three models and the second highest on CogVideoX-1.5.It is the only variant above Base on VideoReward-Overall, HPSv3, and UniPercept across all four models.Source: §6.4 · p. 8, Table 4 · p. 9
  • GAL-Image improves imaging quality on all four models but falls below Base in VideoReward-Overall on every model, with dynamic-degree changes ranging from −4.20% to +32.35%.GAL-VideoReward improves imaging quality and dynamic degree over Base; higher image quality does not necessarily entail motion loss.Source: §6.4 · p. 8

6.5 SCAFFOLD ABLATION

  • Removing the differential-LoRA scaffold changes HunyuanVideo-1.5 dynamic degree from +5.62% to −2.25%, despite higher imaging and aesthetic quality than Base.The unscaffolded variant has a larger VideoReward-Overall increase above Base (+0.289 versus +0.145) but loses the motion gain.Source: §6.5 · pp. 8–9

6.6 DISCUSSION

  • Preference construction uses only HunyuanVideo-1.5, whose dynamic-degree readout has lower within-prompt variance than LTX-Video and CogVideoX-1.5; other backends or combined preferences may yield larger motion gains.The authors also identify dependence on video-reward reliability, limited reward-model choice, and small motion-smoothness regressions under the default reward.Source: §6.6 · pp. 9–10
  • Anchor learning aligns image and video generation upstream without modifying the downstream I2V model, and its transfer results suggest some video-effective anchor properties generalize beyond the training renderer.It is presented as a separate optimization component that could complement I2V architecture and sampler improvements.Source: §6.6 · p. 10

7 CONCLUSION

GAL learns anchors from the quality of the videos they induce, improving results across four frozen I2V models while keeping deployment to one T2I sample and one I2V render. The paper also reports its use of AI tools and points to reproducibility details.

7 CONCLUSION

  • GAL raises VBench average above Base across four frozen I2V models, with higher VideoReward-Overall scores for rendered clips and higher HPSv3 and UniPercept scores for their extracted frames.The reported gains also include dynamic motion, imaging quality, and aesthetic quality.Source: §7 · p. 10
  • GAL constructs video-derived preferences offline and optimizes the upstream T2I LoRA, leaving deployment as one T2I sample followed by one I2V render.Candidate anchors are rendered with a frozen I2V model and scored with a frozen video reward.Source: §7 · p. 10
  • The authors used language models to draft and polish the manuscript, check appendix derivations, and search related literature, but not to design, run, or interpret experiments.The authors state that they reviewed AI-assisted text and derivations and take responsibility for the final content.Source: AI USE STATEMENT · p. 10
  • Training hyperparameters and implementation details are in Appendix C, while evaluation settings and prompt protocols are in Section 6.1 and Appendix D.Source: REPRODUCIBILITY STATEMENT · p. 10

A OBJECTIVE-LEVEL DERIVATIONS

The appendix derives an objective-level gradient and an idealized DPO interpretation, then states their assumptions and limits. It also explains reverse-time SDE sampling for diverse sibling anchors.

A OBJECTIVE-LEVEL DERIVATIONS

  • Under differentiability and an integrable-domination assumption, the chain rule and differentiation under the expectation yield the objective-level pullback gradient.The renderer Jacobian with respect to the anchor links downstream reward sensitivity to anchor changes; GAL does not evaluate this gradient, instead using offline preference labels.Source: A OBJECTIVE-LEVEL DERIVATIONS · p. 16
  • The population DPO target follows when preferences obey the stated logistic model, the log policy ratio is exactly represented, and comparison pairs have full support.This is an objective-level interpretation, not a convergence claim for the implemented optimizer.Source: A OBJECTIVE-LEVEL DERIVATIONS · p. 16
  • The DPO interpretation excludes finite and margin-filtered data, deterministic labels, diffusion-loss surrogates, restricted LoRA, adapter removal at inference, and optimization error.Equation (8) is therefore used only to interpret the objective's direction.Source: A OBJECTIVE-LEVEL DERIVATIONS · p. 16
  • A larger preference margin suppresses noise-induced inversions but retains fewer pairs, and does not establish that one render recovers O(x, p).The margin is treated as a filtering heuristic; robustness is evaluated with independent rendering seeds.Source: A OBJECTIVE-LEVEL DERIVATIONS · p. 16

B SIBLING ANCHOR SAMPLING VIA A MARGINAL-PRESERVING REVERSE-TIME SDE

  • Reverse-time SDE sampling adds trajectory stochasticity beyond the initial latent draw while preserving the probability-flow ODE's time-marginals.Decoding terminal latents with the frozen VAE yields sibling anchors approximating the T2I distribution; the Brownian term supplies within-prompt diversity, with K=4 siblings sampled per prompt.Source: B SIBLING ANCHOR SAMPLING VIA A… · p. 16

C HYPERPARAMETERS, COMPUTE BUDGET, AND ADDITIONAL DETAILS

The appendix documents shared offline DPO-LoRA configurations, reward-specific preference datasets, compute, and implementation choices. The training variants share a candidate corpus but differ in reward labels and, for one ablation, the scaffold.

C HYPERPARAMETERS, COMPUTE BUDGET, AND ADDITIONAL DETAILS

  • The reported reward variants use the same Z-Image-Turbo base, offline DPO-LoRA setup, and step-9000 checkpoint, with rank 64 LoRA and 158,760,960 trainable parameters.The section also specifies shared optimization, precision, parallelism, and image/sequence-size settings.Source: C HYPERPARAMETERS, COMPUTE BUDGET, AND… · p. 17
  • The matched noising context reuses timestep and noise draws across winning and rejected images and across trainable and reference losses, reducing variance in the DPO difference.The implementation checks that winner and loser noise seeds match.Source: C HYPERPARAMETERS, COMPUTE BUDGET, AND… · p. 17
  • Preference variants score a shared pool of 74,124 candidate pairs with their respective video, frame, or anchor-image rewards, then apply a margin of m = 0.05.Pre-split pair counts differ by reward configuration; the training-cluster and effective-batch difference is identified as a minor confound.Source: C HYPERPARAMETERS, COMPUTE BUDGET, AND… · p. 17
  • The scaffold ablation differs from GAL-VideoReward in the presence or absence of the preset adapter, while retaining the same reward and training configuration.The preset adapter is scaffolded into the base via deep-copy before reference cloning.Source: C HYPERPARAMETERS, COMPUTE BUDGET, AND… · pp. 17–18
  • 49,416 candidate records required an estimated 4,100 GPU-hours for HunyuanVideo-1.5 rendering, while each DPO variant used a configured 40,000-step maximum schedule.Per-variant DPO wall-times were not logged; evaluations use the step-9000 checkpoint.Source: C HYPERPARAMETERS, COMPUTE BUDGET, AND… · p. 18
  • Table 5 reports DPO pair counts by absolute reward-margin threshold; all reward variants use m = 0.05, and GAL-VideoReward has 52,084 pairs.Source: Table 5 · p. 18

D PROMPT SOURCES AND VBENCH PROMPT PROTOCOL

The appendix describes disjoint training and evaluation prompt pools and a shared VBench instruction template. VBench uses the official prompt suite, while Kling evaluation uses held-out prompts from Lexica.

D PROMPT SOURCES AND VBENCH PROMPT PROTOCOL

  • Training uses 12,354 canonicalized Lexica prompts, each paired with four SDE-sampled sibling anchors, while the VBench pool uses the official suite.The VBench prompt counts are 93 for imaging quality, aesthetic quality, and overall consistency, and 72 for dynamic degree, subject consistency, and motion smoothness.Source: D PROMPT SOURCES AND VBENCH PROMPT… · p. 18
  • Kling evaluation uses 50 held-out Lexica prompts because Kling v3 Pro is a paid closed-source I2V model.Source: D PROMPT SOURCES AND VBENCH PROMPT… · p. 19
  • Canonicalized prompt-overlap counts are zero among training, VBench, and Kling pools; the VBench pool comes from a different source than training.Source: D PROMPT SOURCES AND VBENCH PROMPT… · pp. 18–19
  • Every VBench evaluation extends official prompts into a complex-motion video-instruction template, with the motion cue fixed by category across variants and the three open-source I2V models.The extension addresses short noun-phrase captions that under-exercise the dynamic-degree axis in image-conditioned I2V.Source: D PROMPT SOURCES AND VBENCH PROMPT… · p. 19

E ANCHOR-SEED SENSITIVITY OF THE DYNAMIC-DEGREE READOUT

Anchor-seed sensitivity varies across I2V models: LTX-Video and CogVideoX-1.5 show larger within-prompt variance, while WAN has the highest dynamic fraction and lowest variance. The probe measures changes in binary dynamic-degree decisions, not all video motion or VBench dimensions.

E ANCHOR-SEED SENSITIVITY OF THE DYNAMIC-DEGREE READOUT

  • Across 72 VBench prompts, 12 sibling anchors per prompt were rendered with each frozen I2V model while its downstream seed and sampling configuration stayed fixed.The dynamic fraction averages binary labels; anchor-seed sensitivity is the average within-prompt variance over the 12 seeds.Source: E ANCHOR-SEED SENSITIVITY OF THE… · p. 19, Table 7 · p. 20
  • WAN has the highest dynamic fraction and smallest within-prompt variance, while LTX-Video and CogVideoX-1.5 show larger variance across anchor seeds.The variance reflects how often seeds change the binary dynamic/static decision for the same prompt; WAN's readout is near-saturated.Source: E ANCHOR-SEED SENSITIVITY OF THE… · pp. 19–20, Table 7 · p. 20
  • This finite-sample probe measures binary dynamic-degree sensitivity, not continuous motion changes, renderer derivatives, or other VBench dimensions and reward variants.Continuous motion can change without crossing the metric's binary threshold.Source: E ANCHOR-SEED SENSITIVITY OF THE… · p. 20

F FAILURE-MODE CASE STUDIES

The case studies report texture-sharpening and camera-motion patterns associated with reduced motion smoothness in selected prompts, without establishing their causes. The per-interval visualization is a qualitative diagnostic proxy, while Table 8 reports reward-model scores for three transfer I2V models.

F FAILURE-MODE CASE STUDIES

  • −0.13% on HunyuanVideo-1.5, −0.19% on LTX-Video, −0.86% on CogVideoX-1.5, and −0.41% on Kling v3 Pro in motion smoothness accompany sharper textures in selected high-frequency prompts.The inspected renders show slightly inconsistent edges across frames; the authors say this does not establish a texture-driven mechanism.Source: F FAILURE-MODE CASE STUDIES · p. 20
  • Selected global-camera-motion prompts show sharper compositions and higher inter-frame contrast co-occurring with less smooth interpolation, but do not isolate composition as the cause.These are case-study observations, not a causal finding.Source: F FAILURE-MODE CASE STUDIES · p. 20

G PER-INTERVAL MOTION VISUALIZATION

  • The per-interval visualization measures residual flow after dominant translational flow is removed, using five contiguous intervals and all decoded adjacent-frame pairs.Its values are a diagnostic proxy that can reflect object motion, parallax, flicker, and flow-estimation error; they are used only to diagnose selected examples, not as aggregate evaluation metrics.Source: G PER-INTERVAL MOTION VISUALIZATION · p. 21

H REWARD-MODEL SCORES ON TRANSFER I2V MODELS

  • Table 8 reports VideoReward-Overall for rendered clips and HPSv3 and UniPercept for extracted frames on LTX-Video, CogVideoX-1.5, and Kling v3 Pro, which were excluded from preference construction.The supplied table caption gives metric scope but not cell values.Source: H REWARD-MODEL SCORES ON TRANSFER I2V… · p. 21, Table 8 · p. 22
Loading 2610.13217v1…