PaperlayerView exact revision on arXiv

Source-linked AI summary

MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing

Gal Fiebelman, Hadar Averbuch-Elor, Sagie Benaim

arXiv:2607.05376v1cs.CVcs.GR

TL;DR

Generating videos that are both temporally long and consistent across multiple viewpoints remains challenging. MV-Forcing combines temporal and view-wise autoregression with a 4D geometric bridge, producing geometrically consistent multi-view videos at arbitrary lengths and viewpoint counts using one few-step student model.

  • Problem

    Long multi-view video generation must preserve continuous scene dynamics and strict 3D consistency across unbounded temporal horizons and arbitrary viewpoints.

  • Method

    MV-Forcing uses autoregressive 4D reconstruction as a geometric bridge between sequential views and applies spatio-temporal self-forcing during distillation.

  • Results

    MV-Forcing maintains robust cross-view synchronization and camera accuracy while scaling to arbitrary viewpoint counts and temporal lengths on synthetic and real-world data.

  • Takeaways & Limitations

    The framework enables spatially consistent multi-view video generation over unbounded temporal horizons and arbitrary viewpoint counts with a single few-step model.

  • Takeaways & Limitations

    Quality still degrades over long horizons, and the two-view teacher does not directly supervise consistency across more than two simultaneous views.

Abstract

from arXiv · show

Recent advances in video diffusion models have enabled either long single-view generation through temporal autoregression, or short multi-view synthesis through bidirectional attention. However, generating long, multi-view consistent videos of dynamic scenes remains unsolved. In this work, we present MV-Forcing, a framework that composes temporal and view-wise autoregression within a single diffusion model by introducing a 4D geometric bridge between sequentially generated views. Our key insight is that an autoregressive 3D reconstruction model naturally interfaces between autoregressively generated views. Given a completed source view, we reconstruct its 3D structure and render a geometric prior of the next target viewpoint, which the diffusion model refines into a high-quality video. To extend generation beyond the teacher's fixed temporal window, we introduce a joint denoising regime where both view slots are initialized from noise during training, enabling temporally unbounded generation. We distill the model via Distribution Matching Distillation with Spatio-Temporal Self-Forcing, closing the train-inference exposure bias gap for both temporal and view-sequential autoregression. Extensive experiments on both synthetic and real-world data demonstrate that MV-Forcing produces geometrically consistent multi-view videos of dynamic scenes at arbitrary lengths and viewpoint counts using a single few-step student model.

1 Introduction

MV-Forcing addresses the unsolved problem of generating long videos that remain temporally coherent and geometrically consistent across arbitrary viewpoints. It composes temporal and view-wise autoregression through a 4D reconstruction bridge, joint denoising, and Spatio-Temporal Self-Forcing.

  • Challenge: Long multi-view video generation must preserve continuous scene dynamics and strict 3D geometric consistency across arbitrary camera trajectories and viewpoints.The task demands simultaneous spatial, temporal, and geometric reasoning over unbounded horizons and arbitrary viewpoint counts.
  • Challenge: Existing approaches extend temporal duration for single views or synthesize short multi-view sequences, but sequential view generation suffers exposure bias and spatial drift without geometric anchoring.Consequently, unbounded, geometrically consistent generation across arbitrary viewpoints remains unsolved.
  • Method: MV-Forcing composes temporal and view-wise autoregression in one model, using an autoregressive 4D reconstruction model as a continuous geometric bridge between generated views.The bridge circumvents reliance on dense bidirectional attention by conditioning the next view on reconstructed geometry from a completed source view.
  • Method: Joint denoising initializes both view slots from pure noise during training, unifying first-view text-to-video generation with view-sequential conditioning and enabling temporally unbounded generation.The regime extends generation beyond the teacher’s fixed temporal window.
  • Evaluation: The framework is evaluated on synthetic and real-world data against SynCamMaster for short sequences and Self-Forcing and ReCamMaster for long sequences.The paper presents MV-Forcing as the first framework for long multi-view video generation.
  • Method: Distribution Matching Distillation and Spatio-Temporal Self-Forcing close the train-inference exposure bias gap for temporal and view-sequential autoregression.Together, they support temporally unbounded, geometrically consistent synthesis.

2 Related Work

Prior work studies autoregressive video generation, multi-view video synthesis, and reconstruction-guided generation as separate routes toward scalable temporal or geometric consistency. MV-Forcing builds on these directions by connecting sequential views through reconstructed and rendered 3D structure.

  • Autoregressive Video Generation: Autoregressive video methods generate frames or frame chunks sequentially, contrasting clean-context Teacher Forcing with independently noised Diffusion Forcing.These formulations address limitations of bidirectional attention, including restricted streaming and scaling.
  • Multi-View Video Generation: Object-centric multi-view methods generate dense orbital views but remain restricted to single-object scenes and fixed cameras, while open-world methods enable multi-camera control or cross-view 3D consistency.The passage cites SV4D, SV4D 2.0, CAT4D, SynCamMaster, and CVD as representative approaches.
  • Reconstruction for Generation: Reconstruction-guided generation renders reconstructed geometry from target viewpoints and conditions a generative model to fill in appearance details.Feed-forward models including DUSt3R, MASt3R, VGGT, and CUT3R predict dense pointmaps and camera poses for this structural-prior paradigm.

3 Method

MV-Forcing composes temporal and view-sequential autoregression in one self-forcing framework, using recurrent 3D reconstruction to provide geometric conditioning between generated views. A distilled few-step student supports autoregressive generation across arbitrary temporal horizons and viewpoint counts, including flexible traversal of the time×view grid.

  • Framework: MV-Forcing composes temporal and view-sequential autoregression within a unified self-forcing framework, using dynamic 3D reconstruction as a geometric bridge between both axes.The framework targets long, multi-view consistent videos with arbitrary viewpoint counts and unbounded temporal horizons.
  • Spatio-Temporal Self-Forcing: Spatio-temporal self-forcing extends temporal self-forcing to view-sequential unrolling, distilling the bidirectional teacher into a causal few-step student via asymmetric DMD.The student learns view-wise autoregression during training rather than relying on jointly denoised views at inference.
  • View-Sequential Generation: Each new view starts from pure noise and is denoised conditioned on the clean preceding view, prompt, and camera parameters, then supplies clean conditioning for the next view.Because each view attends only to its single predecessor, the process is fully autoregressive across viewpoints and extends to arbitrary-length view chains.
  • Geometric Conditioning: CUT3R provides a recurrent, persistent 3D scene state that can be queried from arbitrary virtual cameras to render geometric priors for ungenerated viewpoints and timesteps.This addresses geometric degradation and accumulated 3D errors caused by conditioning each view only on its predecessor.
  • Inference Traversal: The autoregressive structure and accumulated geometric prior allow inference to advance temporally or across views at any block, using temporal KV context and the preceding view’s geometric prior.Generation can follow arbitrary traversal orders over the time×view grid instead of completing one axis before the other.

4 Experiments

MV-Forcing is evaluated on synthetic and real-world long multi-view video generation using visual-quality, camera-accuracy, and cross-view-synchronization metrics. Experiments show stronger geometric consistency than baselines, robustness across views and duration, and remaining limitations in long-horizon quality and real-world supervision.

  • Evaluation Setup: Evaluation uses SynCamVideo with 3,400 synthetic scenes from 10 synchronized cameras, plus real-world Mixkit videos from Open-Sora with minimal finetuning.The synthetic evaluation holds out 100 scenes.
  • Comparisons: MV-Forcing improves camera accuracy and view synchronization over SynCamMaster in short sequences, while visual-quality metrics are slightly lower after distilling the teacher into a few-step causal student.SynCamMaster generates 2 views within a fixed 81-frame window; long-sequence comparisons extend to 3 views and 162 frames.
  • Comparisons: Qualitatively, MV-Forcing preserves geometric consistency across views and timesteps, whereas SF+ReCamMaster suffers late-timestep degradation and SF+ReCamMaster+SF exhibits cross-view drift.The long-sequence comparison covers 3 views across 162 frames.
  • Ablations and Scaling: View-sequential unrolling causes the largest ablation degradation, followed by removing CUT3R; removing accumulation or using manual rendering causes smaller but consistent metric declines.These ablations are evaluated at 3 views and 162 frames.
  • Ablations and Scaling: Metrics remain stable as views increase from 2 to 5 and cross-view metrics remain nearly constant as duration increases from 81 to 648 frames, although CLIP-F declines most noticeably.The authors attribute view-scaling robustness to grounding each new view in the accumulated geometric prior.
  • Limitations: Limitations include predominantly synthetic training, quality degradation over long horizons, and DMD supervision that does not directly supervise consistency beyond two simultaneously generated views.Large-scale real multi-view training and complementary temporal-stability methods are identified as potential improvements.

5 Conclusion … B Additional Details

MV-Forcing combines temporal and view-sequential autoregression through an online dynamic 3D reconstruction model that geometrically bridges generated views and maintains persistent scene state. The paper also provides interactive visualizations covering results, comparisons, ablations, and scaling studies, alongside ethics and funding statements.

  • 5 Conclusion: MV-Forcing composes temporal and view-sequential autoregression within a single generative model for long multi-view video generation.The framework is presented as the paper’s central contribution.
  • 5 Conclusion: An online dynamic 3D reconstruction model bridges sequentially generated views by accumulating persistent state from previously generated content.This state grounds each new viewpoint in the geometry of earlier generated content.
  • 5 Conclusion: Self-Forcing is extended to the view axis to mitigate exposure bias in sequential multi-view generation.
  • 5 Conclusion: The method builds on publicly available open-source models and datasets.The ethics statement characterizes the contribution as long multi-view video generation without introducing new generative content beyond existing open-source text-to-video models.
  • 5 Conclusion: The research was supported by The Israel Science Foundation and EuroHPC JU.The cited support includes grant No. 2416/25 and Application ID EHPC-DEV-2025D08-098.
  • A Interactive Visualizations: Interactive visualizations present multi-view results across diverse scenes and real-world generalization examples.They are available at https://galfiebelman.github.io/mv-forcing/supp/index.html.
  • A Interactive Visualizations: The visualizations also include baseline comparisons, component ablations, and view and temporal scaling results.

B.1 Implementation Details

MV-Forcing uses a causalized Wan2.1-T2V-1.3B student with SynCamMaster’s MVS module and trains it in two stages for DMD distillation with spatio-temporal self-forcing. Training uses 16 NVIDIA A100 65GB GPUs, with joint denoising probability p = 0.3 and CUT3R frozen throughout.

  • Architecture: The student inherits the Wan2.1-T2V-1.3B backbone, replaces bidirectional temporal attention with causal blockwise attention, and retains SynCamMaster’s MVS module in every transformer block.The temporal block size is K = 3 latent frames.
  • Training: Training has two stages: ODE initialization followed by DMD distillation with spatio-temporal self-forcing, using Adam on 16 NVIDIA A100 65GB GPUs with effective batch size 16.Both stages use batch size 1 per GPU, with β1 = 0 and β2 = 0.999.
  • Training: 1,600 iterations are used for the second-stage asymmetric DMD training, with generator and critic learning rates of 2 × 10^-6 and 4 × 10^-7, respectively.This stage takes approximately 26 hours.
  • Training: The joint denoising probability is set to p = 0.3, while CUT3R remains frozen throughout both training stages.The joint denoising probability applies during DMD distillation with spatio-temporal self-forcing.

B.2 Real-World Extension Details

The real-world extension adapts ReCamMaster to non-overlapping target viewpoints by finetuning it on arbitrary viewpoint pairs and training only the view-sequential conditioning path.

  • Teacher adaptation: ReCamMaster assumes the generated video shares its first frame with the reference, unlike the target viewpoint setting with no overlapping first frame.This mismatch motivates adapting the real-world teacher for multi-view generation.
  • Teacher adaptation: 2,000 iterations of finetuning on SynCamVideo with arbitrary viewpoint pairs adapt ReCamMaster to the non-overlapping setting.The same finetuned model is used for the baseline comparisons in Sec. B.6.
  • Training configuration: The real-world training sets joint denoising probability p=0 and trains only the view-sequential conditioning path using the DMD loss.The finetuned ReCamMaster serves as the data score s_data during training.

B.3 Geometric Prior Details

The geometric prior accumulates reconstructed information across generated views and temporal blocks, then uses confidence to weight its reliability. Visualization shows that integrating additional views improves scene structure and confidence.

  • Accumulation Order: CUT3R’s persistent state accumulates decoded frames from prior views and earlier temporal blocks before raymaps render the next view’s camera sequence.At view k and temporal block t, the state includes views 0 through k−1 and view k’s earlier blocks.
  • Confidence Maps: Per-pixel confidence maps indicate reconstruction certainty, enabling the Conv3d layer to weight the geometric prior according to local reliability.Overlapping regions receive high confidence, while occluded or unobserved regions receive low confidence.

B.4 Runtime Analysis

MV-Forcing’s causal student enables fast inference with modest geometric-prior overhead, while runtime scales linearly with views and temporal length and peak memory remains constant. It achieves approximately 5× speedup over SynCamMaster in the short-sequence setting while supporting unbounded scaling.

  • Runtime: Approximately 20 seconds are required for a single 81-frame view with 4-step denoising on one NVIDIA A100 65GB GPU.The measurement concerns the student forward pass.
  • Runtime: Approximately 30 seconds per view transition are added by the CUT3R update and geometric-prior query.This overhead is measured on the same single NVIDIA A100 65GB GPU setup.
  • Scaling: Approximately 5× speedup is achieved over SynCamMaster for the short-sequence setting, while supporting unbounded temporal and view scaling.SynCamMaster requires approximately 6 minutes per 81-frame chunk because of 50-step bidirectional denoising.
  • Scaling: Runtime scales approximately linearly with view count and temporal length, while peak VRAM remains constant at 23 GB across configurations.The scaling measurements are reported across different view and temporal configurations on one NVIDIA A100 65GB GPU.

B.5 Evaluation Details · B.6 Baseline Details

The evaluation averages visual, camera-accuracy, and cross-view synchronization metrics over 100 held-out scenes and three camera configurations. Baselines include SynCamMaster, Self-Forcing/ReCamMaster combinations, and an in-domain finetuned SF(ft) variant for synthetic evaluation.

  • B.5 Evaluation Details: Evaluation uses 100 held-out scenes across azimuthal rotation, elevation change, and distance variation, averaging all metrics across configurations and scenes.The same protocol and metrics apply to SynCamVideo and Mixkit; visual metrics include FID, FVD, CLIP-T, and CLIP-F.
  • B.5 Evaluation Details: Camera accuracy estimates relative pose from GIM sparse matches and reports RotErr and TransErr against ground-truth poses.Matches are sampled every 4th pixel frame; 5,000 sparse matches support essential-matrix estimation via RANSAC.
  • B.5 Evaluation Details: Cross-view synchronization is measured with Mat. Pix, CLIP-V, and FVD-V across consecutive views and timesteps.Mat. Pix counts high-confidence matching pixels, CLIP-V averages pairwise CLIP cosine similarity, and FVD-V evaluates stacked cross-view clips padded to 10 frames when needed.
  • B.6 Baseline Details: Baseline comparisons use the authors’ official implementations and configurations described for reproducibility.The baseline suite includes SynCamMaster and Self-Forcing/ReCamMaster combinations.
  • B.6 Baseline Details: SynCamMaster uses Wan2.1-T2V-1.3B with 50 denoising steps and guidance scale 5.0 to jointly generate 2 views in an 81-frame window.It uses bidirectional temporal and cross-view attention.
  • B.6 Baseline Details: SF+ReCamMaster generates an arbitrarily long first view with Self-Forcing, then re-renders additional views with ReCamMaster from target camera trajectories.SF+ReCamMaster+SF further temporally extends each additional view beyond the initial ReCamMaster chunk using Self-Forcing and ReCamMaster output as context.
  • B.6 Baseline Details: SF(ft) finetunes Self-Forcing on SynCamVideo for 1,000 iterations, scores only V0, and replaces Self-Forcing in both baselines for synthetic evaluation.The finetuning replaces the real-world teacher with SynCamMaster to bridge the domain gap.

C Failure Case Analysis

MV-Forcing failures fall into three categories: artifacts in the first view propagate autoregressively, while extreme camera displacement and motion corrupt geometric priors and cross-view consistency.

  • Failure Case Analysis: The analysis identifies three categories of failure cases in Fig. 5.These are generation backbone quality, extreme camera displacement, and extreme motion.
  • Generation Backbone Quality: Artifacts in text-only first view V0 are faithfully propagated to subsequent views across views and timesteps.When V0 is artifact-free, subsequent views remain clean; a stronger generation backbone or teacher could mitigate this failure.
  • Extreme Camera Displacement: Large source-target viewpoint differences create depth ambiguity and incorrect spatial correspondences, displacing sofa geometry onto the floor.The geometric prior is unreliable in regions unobserved from previous views, especially at steeper viewing angles.
  • Extreme Motion: Fast or complex motion can make CUT3R reconstruct incorrect limb geometry, producing inconsistent numbers of visible legs across views.The corrupted geometric prior provides unreliable spatial anchors; recovery could rely on the previous view ˆzk−1, but such cases are highly out-of-distribution.

D Additional Comparisons and Ablations

Additional comparisons show MV-Forcing preserves visual quality and cross-view consistency in short and synthetic long sequences, outperforming drift-prone baselines. Ablations and scaling studies identify geometric priors and view unrolling as important, while consistency remains stable across more views and longer durations.

  • Short Sequence Comparisons: At 2 views and 81 frames, MV-Forcing matches SynCamMaster’s visual quality while achieving stronger cross-view consistency.The comparison highlights cross-view inconsistencies in SynCamMaster that the accumulated geometric prior overcomes.
  • Synthetic Long Sequence Comparisons: At 3 views and 162 frames, MV-Forcing maintains consistent appearance and geometry across views and timesteps, unlike SF(ft)+ReCamMaster and SF(ft)+ReCamMaster+SF.SF(ft)+ReCamMaster degrades beyond its 81-frame window, producing later-timestep artifacts.
  • Component Ablations: Removing view-sequential unrolling or the CUT3R geometric prior causes the most visible degradation in cross-view consistency at 3 views and 162 frames.Without view unrolling, background elements shift across views; without CUT3R, occluded backgrounds change between viewpoints.
  • View Scaling: From 2 to 5 views at 81 frames, visual quality and geometric consistency remain stable as the view count increases.The accumulated geometric prior grounds each new view in the full 4D structure observed so far.
  • Temporal Scaling: Across an 8× duration increase from 81 to 648 frames at 2 views, MV-Forcing maintains consistent appearance, motion, and cross-view synchronization.The temporal-scaling results use evenly spaced relative positions across the generated sequence.
Loading 2607.05376v1…