PaperlayerView exact revision on arXiv

Source-linked AI summary

Video Generation Models: A Survey of Post-Training and Alignment

Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, Tianzhi Li, Fei Tao, Pooyan Fazli

arXiv:2610.00812v1cs.CVcs.AIcs.LG

TL;DR

Pretrained video models may not reliably follow intent, maintain temporal coherence, or satisfy physical constraints, while temporal errors, competing objectives, and limited supervision complicate alignment. This survey reviews post-training approaches and classifies them by alignment signals and method family. It concludes that recent approaches have improved controllability, temporal coherence, and alignment with user intent, while identifying unresolved challenges.

  • Problem

    Pretrained video models may fail to follow user intent or maintain temporal coherence and physical plausibility, while temporal errors accumulate and reliable temporal supervision is scarce.

    Source: §1 · pp. 1–2

  • Method

    The survey reviews post-training methods and organizes them by implicit or explicit alignment and four method families spanning training and inference.

    Source: §1 · pp. 4–5

  • Results

    The survey reports that recent approaches have improved controllability, temporal coherence, and alignment with user intent.

    Source: §10 · p. 49

  • Takeaways & Limitations

    The taxonomy provides a unified view of how post-training techniques shape alignment across training and inference.

    Source: §1 · p. 4

    Video-level reward signals can miss localized temporal failures, including sliding or object interpenetration, that reduce physical plausibility.

    Source: §9.3 · p. 47

Abstract

from arXiv · show

Video generation has rapidly progressed from short, low-quality clips to high-resolution, long-duration sequences with complex spatiotemporal dynamics. Despite strong generative priors learned through large-scale pretraining, pretrained video models often fail to reliably follow human intent, maintain temporal coherence, or satisfy physical and safety constraints. Compared with image and text generation, alignment in video generation presents unique challenges, including error accumulation over time, motion-appearance coupling, multi-objective trade-offs, and limited supervision for temporal properties. These challenges motivate systematic post-training strategies that adapt pretrained models without retraining them from scratch. In this survey, we present the first comprehensive review of post-training and alignment in video generation models. We frame post-training as a unifying framework and distinguish between implicit alignment and explicit alignment based on how alignment signals are enforced. From this perspective, we organize existing approaches into four broad categories: supervised fine-tuning methods, self-training and distillation methods, preference- and reward-based methods, and inference-time methods. This taxonomy provides a coherent view of how alignment signals shape model behavior across both training and deployment. Beyond methodological advances, we review commonly used datasets, benchmarks, and evaluation practices, and discuss open challenges such as scalable reward design, long-horizon temporal consistency, stability-expressiveness trade-offs, and safety-aware generation. This survey aims to provide a structured conceptual foundation and practical guidance for advancing controllable and reliable video generation models.

1. Introduction

Video generation has advanced to longer, higher-quality sequences, but pretrained models still struggle with intent, temporal coherence, and physical constraints. This survey organizes post-training alignment by how its signals are enforced and reviews methods, benchmarks, and evaluation practices.

1. Introduction

  • Pretrained models can fail to follow intent, preserve identity over long videos, maintain stable motion, or produce physically realistic interactions despite large-scale pretraining.The survey frames these failures as a mismatch between pretraining objectives and real-world behavioral requirements.Source: §1 · p. 2
  • Video alignment is difficult because frame-level errors accumulate, objectives can conflict, and reliable temporal supervision is scarce and costly.Proxy metrics and learned evaluators may introduce bias.Source: §1 · p. 3
  • Implicit alignment shapes behavior without directly evaluating correctness, whereas explicit alignment uses evaluative signals; alignment lies on a spectrum rather than a strict binary.The distinction concerns how alignment is enforced, not when or where optimization occurs.Source: §1 · p. 4
  • The survey groups methods into supervised fine-tuning, self-training and distillation, preference- and reward-based methods, and inference-time methods.The first two primarily provide implicit alignment, preference and reward methods provide explicit alignment, and inference-time methods can support either.Source: §1 · p. 4, Figure 3 · p. 6
  • The survey reviews post-training methods and their video-specific adaptations, proposes an alignment taxonomy, and summarizes datasets, benchmarks, and evaluation protocols.The taxonomy organizes approaches by optimization mechanisms and alignment roles.Source: §1 · p. 5

2. Preliminaries: Video Generation Models and Alignment Dimensions

This section sets up video generation as conditional T2V, I2V, or V2V modeling and frames alignment as a multi-objective problem. It outlines the base models and behavioral dimensions relevant to post-training.

2.3. Alignment Dimensions for Video Generation

  • Alignment must balance instruction adherence, temporal coherence, motion realism, perceptual quality, and safety rather than optimize likelihood alone.These objectives can conflict, including stability and expressiveness.Source: §2 · p. 7, §2.3 · p. 10

2.1. Video Generation Problem Setting

  • T2V generates from text and a learned prior, while I2V extends a reference image with dynamics while preserving its identity and visual details.T2V starts from latent noise; I2V conditions on a text prompt and reference image.Source: §2.1 · p. 7
  • V2V edits a source video under an instruction while preserving its layout, trading off edit strength against structural consistency and temporal coherence.Conditioning injection or inversion-based guidance can anchor layouts while changing semantics.Source: §2.1 · p. 8

2.2. Base Models

  • Modern video DiTs compress video into spatiotemporal latents, denoise conditioned tokens, and decode the predicted latents; compression reduces sequence length but obscures fine temporal detail.Text conditioning enters the denoising backbone, a primary interface for alignment.Source: Figure 5 · p. 9, §2.2 · p. 9
  • Spatiotemporal tokens use positional encodings for temporal-length flexibility, while long-video post-training may need explicit position handling to avoid drift or mismatch.Flow Matching learns a latent-space velocity field mapping noise to data.Source: §2.2 · p. 9
  • Autoregressive models generate discretized spatiotemporal tokens by causal next-token prediction, allowing standard PPO or DPO on token logits; most surveyed methods instead focus on continuous diffusion trajectories.The survey attributes this focus to the current open-source landscape and recent post-training advances.Source: §2.2 · p. 10

2.3. Alignment Dimensions for Video Generation

  • Instruction alignment covers complex and multi-step prompts plus structured controls such as camera paths, depth, poses, or layouts; temporal alignment targets flicker, unstable backgrounds, shape changes, and identity drift.Identity consistency matters across viewpoint changes and partial occlusion, especially in long-form or personalized generation.Source: §2.3 · p. 10
  • Motion alignment seeks expressive realistic actions while respecting gravity, collisions, object permanence, and cause and effect; aesthetic and safety goals include clarity, human preferences, and suppression of harmful content.The survey also describes refusal of unsafe requests as part of responsible generation behavior.Source: §2.3 · p. 11

3. Supervised Fine-tuning Methods

This section surveys supervised fine-tuning approaches that adapt pretrained video models through targeted supervision, covering instruction following, domain specialization, controllability, and personalization. It also describes application-specific personalization for human and product videos and data pipelines that add structured semantic signals and scalable supervision beyond raw video–text pairs.

3.1. Preliminaries: A Unified View of Supervised Objectives

  • A unified supervised objective maps conditioning inputs to target videos using model-specific losses; parameter-efficient tuning limits updates, while regularizers can preserve priors, temporal consistency, and robustness.For diffusion- and flow-based models, losses may use denoising, noise prediction, latent reconstruction, or velocity matching.Source: §3.1 · p. 12

3.2. Instruction and Prompt-following Fine-tuning

  • Instruction fine-tuning maps curated natural-language directives to videos; methods scale supervision with synthesized, reward-reweighted data and multimodal prompt-video pipelines [45] [46].GigaVideo-1 uses LLM-generated prompts and frozen-MLLM feedback to reweight synthesized samples; VIMI uses retrieval-augmented examples and two-stage multimodal training.Source: §3.2 · pp. 12–13
  • Image-conditioned prediction, trajectory control, and instruction-guided editing add visual grounding and motion structure, while parameter-efficient tuning and computation-aware designs target specialization costs [48] [49] [50] [53] [54] [55] [57] [60] [61].Approaches include image-grounded future-frame prediction, optical-flow-based editing, LoRA specialization, pruning, and attention optimizations.Source: §3.2 · pp. 13–14

3.3. Domain Adaptation and Specialization

  • Domain adaptation addresses specialized-distribution shifts with long-context scene tuning and data-scarce medical specialization, alongside physical supervision for structures underrepresented in generic video-text data [62] [63] [68] [69].LCT learns cross-shot consistency from scene-level data; Mission Balance separates spatial fidelity from temporal dynamics for rare medical events.Source: §3.3 · p. 14
  • Retention-aware adaptation counters forgetting by preserving temporal priors through correlated noise, neighboring-frame feature alignment, and temporally structured conditioning under limited data [71] [72] [73].These methods target temporal degradation and motion-structure collapse while retaining general spatiotemporal priors.Source: §3.3 · p. 15

3.4. Multi-conditioning and Controllability

  • Structured motion cues such as trajectories [30], camera paths [92], bounding-box trajectories, and part-level drag signals [119] enable temporal, viewpoint, object, and articulation control while preserving appearance, identity, and spatial coherence.Source: §3.4 · pp. 16–17
  • Programmatic interfaces translate prompts into scene scripts or selected plans for long-horizon control, while audiovisual post-training targets synchronization, semantic consistency, and unimodal quality [96, 97, 289].VideoStudio executes a multi-scene program; VideoAgent refines and selects plans, while Apollo uses aligned audio-video training and constrained multitask optimization.Source: §3.4 · p. 17

3.5. Personalization and Style Adaptation

  • Lightweight adapters, synthetic identity pairs, and identity-motion-aware training support subject-consistent generation; portrait, character, and multimodal systems extend personalization across speech, pose, and input modalities [31] [98] [102] [103] [104] [105] [106] [107].The approaches include identity-preserving portrait animation, illustrated-character consistency, reference-image-conditioned human motion, and generation conditioned on audio, images, video, and text.Source: §3.5 · p. 18
  • Personalization targets try-on, human–product demonstrations, and mobile creation while balancing interaction realism, product fidelity, stylistic consistency, and deployment efficiency.DreamVVT [108] aligns garment appearance with human motion and body structure; DreamActor-H1 [109] preserves human dynamics and product details; MobileVidFactory [110] enables personalized video creation under resource constraints.Source: §3.5 · pp. 18–19

3.6. Data Construction and Curation Pipelines

  • Data pipelines add semantic, object-level, motion, human, and audio–visual signals to guide temporal modeling and compositional generation.DreamVE [112] makes editing intent explicit, while VC4VG [111] optimizes textual supervision; LoVoRA [113] and MATRIX [114] provide object tracks, Phantom [115] and Puppet-Master [119] provide part-level motion signals, HuMo [116] and TASTE-Rob [117] use pose and hand–object cues, and audio-driven pipelines provide synchronization signals [292] [118] [120].Source: §3.6 · p. 19
  • Scalable labeling replaces costly dense annotation with synthetic labels, learned feedback, and supervision organized around realistic user intents.LinkTo-Anime [121] renders optical-flow motion labels from controllable animation states, while VideoScore learns reusable automatic feedback; user-intent-driven organization reduces annotation overhead [122].Source: §3.6 · p. 19

4. Self-training and Knowledge Distillation Methods

Self-training and distillation refine pretrained video models using generated targets, teacher guidance, or auxiliary signals rather than relying entirely on newly curated human-labeled data. The section organizes these methods into unified objectives, self-training and test-time training, and knowledge distillation.

4. Self-training and Knowledge Distillation Methods

  • Reusing generated targets, teachers, and auxiliary signals offers scalable refinement when aligned supervision is limited, costly, or noisy.The section frames robustness and inference efficiency as additional motivations for these methods.Source: §4 · p. 21

4.1. Preliminaries: A Unified View of Self-Training and Distillation

  • Self-training applies pseudo-supervision to unlabeled or weakly labeled inputs, while distillation trains a student to match a teacher using a strategy-specific discrepancy.Pseudo-targets may be generated videos, latent trajectories, denoising targets, or other intermediate signals; alignment is transferred indirectly rather than through direct evaluative feedback.Source: §4.1 · p. 21

4.2. Self-training and Test-time Training

  • Test-time methods adapt lightweight parameters, embed recurrent learning in temporal modules, or refine generation offline from self-generated trajectories and errors.Zo3T aligns intermediate frame features during denoising; recurrent TTT layers carry long-range temporal information; VideoAgent reuses successful robot trajectories, while SVI recycles estimated errors.Source: §4.2 · pp. 21–22

4.3. Knowledge Distillation

  • Diffusion-to-autoregressive distillation compresses iterative sampling into a few autoregressive steps for faster long-horizon or streaming generation.CausVid uses distribution matching and ODE-based student initialization; Rolling Forcing jointly denoises consecutive frames and uses attention sinks and extended-window distillation.Source: §4.3 · p. 23
  • Trajectory-level distillation aligns teacher and student behavior across intermediate diffusion states, with variants targeting temporal consistency, error accumulation, diversity, or detail.SwiftVideo matches velocity fields over time, while rCM makes student self-consistency primary and uses teacher score distillation as an auxiliary regularizer.Source: §4.3 · p. 23
  • Adversarial and hybrid distillation adds discriminator feedback to consistency or distribution-matching objectives to address quality, stability, and few-step degradation.DOLLAR combines distribution matching, consistency-based distillation, and latent reward optimization to alleviate mode collapse and fidelity degradation.Source: §4.3 · p. 23
  • Distillation strategies also use progressive resolution and temporal-complexity curricula, self-generated histories, and text-encoder compression.The section describes text-encoder distillation as reducing model size and inference cost while preserving semantic alignment.Source: §4.3 · pp. 23–24

5. Preference- and Reward-based Methods

Preference- and reward-based methods steer pretrained video generators through comparative or evaluative feedback, addressing alignment goals that are difficult to encode as fixed supervised targets. The section reviews optimization paradigms, reinforcement learning, preference construction, and video-specific reward design.

5. Preference- and Reward-based Methods

  • These methods optimize evaluative signals for competing goals such as semantic fidelity, temporal coherence, physical plausibility, identity consistency, and safety.The section groups them into reinforcement learning, preference-based optimization, and video reward modeling.Source: §5 · p. 25

5.1. Preliminaries: Optimization Paradigms

  • PPO, DPO, and GRPO provide representative optimization paradigms, unified here for autoregressive and diffusion-based video generators.Diffusion objectives commonly operate over trajectories or denoising steps because final-video likelihood is not directly tractable; GRPO uses relative comparisons within sampled groups to avoid explicit reward modeling.Source: §5.1 · pp. 25–26

5.2. Reinforcement Learning for Video Generation

  • RL treats generation as sequential decision-making with long-horizon video objectives, supporting trajectory-level optimization for temporal consistency, physical plausibility, and structured constraints.Examples include system-level joint reasoning-to-generation alignment [140], process-level physics or camera rewards [142] [143], and stability- or identity-focused objectives [145] [148].Source: §5.2 · pp. 26–28

5.3. Preference-based Optimization for Video Alignment

  • Video preference methods construct pairwise or structured comparisons to optimize alignment directly, including automated ranking [27], real-versus-edited pairs [151], and adaptive references [152].DenseDPO [156] controls motion bias with segment-level labels on pairs sharing motion structure, while AlignHuman [157] separates motion and fidelity supervision across denoising intervals; PhysHPO [158] organizes preferences by instance, state, and motion.Source: §5.3 · pp. 28–29

5.4. Video Reward Modeling

  • Video reward models must capture temporal dynamics, motion consistency, and long-range coherence, while avoiding reward hacking from coarse or imperfect signals.VideoReward [24] models visual quality, motion quality, and text-video alignment; DenseGRPO estimates step-wise reward gains, SoliReward [325] uses lower-noise annotations and regularized preference learning, and Diffusion-DRF [326] supplies aspect-structured feedback. Physics-aware examples include measurable proxies [168], learned physical rewards [153], and geometric consistency [324].Source: §5.4 · pp. 29–30

6. Inference-Time Methods

Inference-time methods use post-trained signals to steer, constrain, or refine video generation during sampling without further parameter updates. The section distinguishes guidance-based alignment from iterative and staged refinement.

6. Inference-Time Methods

  • Inference-time methods consume post-trained signals during sampling to address residual errors or deployment-specific control needs without updating model parameters.They steer trajectories, enforce semantic or physical constraints, or regulate trade-offs among alignment goals.Source: §6 · p. 32

6.1. Guidance-based Alignment

  • Guidance methods alter denoising trajectories to control semantics, structure, motion, or physical plausibility while keeping backbone parameters fixed.CFG strengthens conditioning with a trade-off against diversity or artifacts; other approaches intervene in latent trajectories [38] [171] or use auxiliary semantic and physical feedback [173] [174] [175].Source: §6.1 · pp. 32–33

6.2. Iterative Refinement and Self-editing

  • Iterative refinement corrects latent states or plans through feedback, while cascaded refinement separates coarse motion and layout from later detail refinement.DragVideo [39] iteratively applies motion supervision, MotionAgent revises motion plans, and iDiT-HOI [180] uses two stages for coarse interaction structure and temporally coherent hand-object dynamics; cascaded stages trade iterative flexibility for stability and predictable cost.Source: §6.2 · pp. 33–34

7. Cross-Family Comparison and Multi-stage Pipelines

The four post-training families are complementary rather than mutually exclusive, and systems often combine them in sequential stages. Their usefulness depends on supervision, target behavior, and when alignment is applied.

7. Cross-Family Comparison and Multi-stage Pipelines

  • The survey organizes methods into supervised fine-tuning, self-training and distillation, preference- and reward-based optimization, and inference-time methods, which can serve complementary roles in one system.The taxonomy identifies the primary mechanism driving behavioral change rather than requiring mutually exclusive categories.Source: §7 · p. 35

7.1. How Architecture Shapes Post-training Interfaces

  • Backbone architecture shapes how alignment objectives are implemented, but does not make a post-training family inherently architecture-specific.Autoregressive models more naturally express token-level policies, whereas diffusion models often use latent trajectories, denoising steps, or sampling-time guidance; the survey’s diffusion-heavy coverage reflects the current backbone landscape.Source: §7.1 · p. 35

7.2. When Different Post-training Families Are More Appropriate

  • Supervised tuning suits available paired targets, self-training and distillation suit scarce or noisy supervision, reward methods suit evaluative goals, and inference-time methods suit settings where retraining is impractical.Reward methods depend on signal quality, while inference-time approaches work best with strong pretrained reward or guidance components.Source: §7.2 · pp. 35–36

7.3. Family Intersections and Multi-stage Composition

  • Cross-family combinations most often form sequential pipelines, assigning adaptation, correction, evaluative refinement, or compression to distinct stages.Examples include supervised tuning followed by DPO [33], distillation followed by reward fine-tuning [134], and Seedance 1.0’s sequence of supervised fine-tuning, video-reward RL, and multi-stage distillation [138].Source: §7.3 · pp. 36–37

8. Datasets, Benchmarks, and Evaluation Protocols

The section reviews how alignment datasets encode objectives, how benchmarks isolate evaluation dimensions, and how protocols measure them. Its comparisons show that post-training gains are dimension-specific and should be read in light of heterogeneous evaluation settings.

8.1. Post-training Datasets

  • Post-training datasets target instruction following, temporal consistency and identity, physical reasoning, or preferences, with examples spanning TIP-I2V, OpenHumanVid, WISA-80K, and SafeSora.These datasets provide signals beyond the coverage-oriented, diverse pretraining corpora.Source: §8.1 · pp. 38–39

8.2. Benchmarks by Alignment Dimensions

  • Benchmarks separate instruction adherence, long-horizon temporal coherence, and physical plausibility, including evaluations of controllability, identity-action binding, and actionable dynamics.These dimensions capture complementary aspects of alignment rather than a single target.Source: §8.2 · pp. 39–40

8.3. Evaluation Protocols and Metrics

  • Evaluation combines automated metrics, learned evaluators, and human judgments; no single protocol captures every alignment dimension.Automated metrics scale, learned evaluators aggregate perceptual and semantic cues, and human evaluation remains the reference standard; hybrid pipelines use them for screening and validation.Source: §8.3 · pp. 40–42

8.4. Benchmark-wise Quantitative Comparison of Post-training Methods

  • Reported benchmark scores are heterogeneous evidence, not a universal ranking, because methods differ in tasks, base models, sampling budgets, and evaluation subsets.Tables 7–9 summarize available reported results on VBench, VBench2, and physical-plausibility benchmarks; missing entries mean scores were not reported.Source: §8.4 · pp. 42–43, Table 7 · p. 43, Table 8 · p. 44
  • Smooth temporal evolution does not guarantee strong motion generation: VBench models with similar temporal consistency or motion smoothness can differ substantially in Dynamic Degree.The tables provide benchmark-wise evidence, not a universal ranking of post-training categories.Source: Table 7 · p. 43, §8.4 · p. 44
  • Supervised fine-tuning results are most informative for dimensions corresponding to available supervision, while self-training and distillation can transfer teacher behavior or reduce inference cost.Perceptual scores alone do not fully capture efficiency factors such as sampling steps, latency, and deployment costs.Source: §8.4 · pp. 44–45
  • Preference- and reward-based results are most relevant when evaluation matches the optimized signal, but gains on one dimension may leave other properties unchanged or degraded.Inference-time methods steer sampling without updating parameters and require attention to guidance conflicts, external critics, and prompt-specific choices.Source: §8.4 · p. 45
  • Targeted benchmarks are needed alongside broad suites because methods often optimize a specific goal, and prompt adherence can coexist with physical violations.VideoPhy and VideoPhy2 separate semantic adherence from physics consistency.Source: §8.4 · p. 45

9. Challenges and Future Directions

The section identifies limits across post-training approaches, from coupled appearance and motion representations to unstable optimization and incomplete evaluation. It calls for more efficient and verifiable methods, controlled comparisons, and stronger safety-aware evaluation and deployment.

9.1. Supervised Fine-tuning

  • Shared spatiotemporal representations make appearance–motion disentanglement difficult, while short-clip tuning and costly long-context methods constrain long-horizon, multi-stage instruction following.Adapting motion can alter identity; movements, details, and backgrounds may drift over time.Source: §9.1 · p. 46

9.2. Self-training and Distillation

  • Self-training can reinforce biases and accumulate early video errors, while few-step distillation can harm temporal smoothness and motion continuity.The section points to verifier-guided reuse of samples and trajectory-aware distillation as possible directions.Source: §9.2 · pp. 46–47

9.3. Preference-based and Reinforcement Learning

  • Holistic video-level rewards can miss localized physical failures and favor conservative motion; costly sampling and long, high-dimensional trajectories also undermine reinforcement-learning efficiency and stability.Suggested directions include physics-grounded, segment-level rewards and off-policy or latent-space optimization.Source: §9.3 · p. 47

9.4. Inference-Time Alignment

  • Inference-time guidance can be task-specific, conflicting, and computationally costly, limiting flexibility and interactive use.The section proposes agent-like monitoring and test-time search or planning as directions for structured control.Source: §9.4 · pp. 47–48

9.5. Evaluation and Benchmarking Challenges

  • Most studies lack controlled cross-paradigm comparisons, while alignment objectives can trade motion, identity, diversity, and long-term coherence against other gains.The section calls for matched protocols and systematic failure analysis, including beyond controlled benchmarks.Source: §9.5 · p. 48
  • Video-specific safety post-training remains limited, and safety risks are not fully captured by standard quality evaluation alone.The section also describes safeguards such as unsafe-generation blocking, watermarking, and forensic feedback.Source: §9.5 · pp. 48–49

10. Conclusion

The survey frames video generation post-training as a shift from pure pretraining toward targeted alignment and optimization. It reviews four approach families, summarizes progress and unresolved challenges, and points to efficiency, stronger grounding, and training–inference integration as future directions.

10. Conclusion

  • The survey reviews supervised fine-tuning, preference- and reward-based methods, self-training, and inference-time control, while identifying alignment and optimization challenges that remain.The conclusion highlights appearance–motion coupling, long-horizon instruction following, reward limitations, and reinforcement-learning inefficiency and instability.Source: §10 · p. 49
Loading 2610.00812v1…