Learning to Correct Geometry in Generated Videos

1 Introduction

• Sparse viewpoint-change supervision in web video leaves models liable to confuse camera motion with scene dynamics and deform geometry instead of preserving stable 3D layout.Existing post-hoc refiners mainly improve appearance, while post-hoc correction of scene geometry under camera motion remains comparatively underexplored.Source: §1 · p. 2
• The paper identifies post-hoc geometry correction as a gap because generator-level geometry methods require modifying models, while existing refiners focus on pixel-level restoration.The introduction also attributes camera-induced distortions to insufficient viewpoint-change supervision and presents VG-Pairs and VGC as its response.Source: §1 · p. 2
• VG-Pairs pairs real videos with diverse camera motion and pose annotations with synthetic distortions, providing geometry-consistent targets with similar semantics and camera motion.Trajectory-adaptive keyframes are inserted at abrupt pose changes to generate shorter subsegments that are later stitched together.Source: §1 · p. 2
• VGC is a post-hoc video-to-video diffusion model that restores structural coherence without regenerating videos from scratch, preserving scene content and object appearance.Optical flow serves as an auxiliary cue to separate globally coherent camera-induced motion from local object dynamics.Source: §1 · p. 2
• Scaling VGC to larger-capacity models extends its capability toward dynamic videos with human and object motion, moving toward general video refinement.The stated main focus remains correcting geometry in static scenes under camera motion.Source: §1 · p. 2

2 Related Work

• Geometry-focused approaches add geometric supervision or spatial priors during training, optimize post-training alignment or rewards, or incorporate explicit 3D structure into generation.Examples include GeoVideo [9], Geometry Forcing [10], VideoGPA [16], and World-consistent Video Diffusion [12].Source: §2 · p. 3
• VGC differs by correcting geometry post hoc with paired supervision from real videos, without modifying the source generator and with transfer across model families.This contrasts with methods that address geometry at the generator or training level.Source: §2 · p. 3
• Video refinement methods improve restoration, resolution, spatial and temporal quality, or motion plausibility, but largely overlook scene-level geometric consistency under camera motion.Examples include Upscale-A-Video [7], VEnhancer [15], and ReVision [23].Source: §2 · p. 3

3 Method

• VGC is trained as a separate video-to-video diffusion model on distorted-input/geometry-consistent-target pairs, with optical flow as an auxiliary cue for disentangling camera motion from scene dynamics.At inference, it computes flow from the distorted input and can refine videos from different generators without modifying their source models.Source: §3 · p. 3, §3.3 · p. 6

3.1 Paired Supervision from Videos with Camera Motion

• Trajectory-adaptive keyframes divide clips into shorter segments so synthesized distortions remain close to the target camera path while retaining geometric errors.Keyframes are placed at trajectory inflections; clips with larger accumulated motion are divided into shorter intervals, while short or nearly linear trajectories use one segment.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Motion Disentanglement with Optical Flow

• Optical flow from the distorted input supplies motion structure: globally coherent fields tend to indicate camera motion, while local deviations tend to indicate object motion.The same input-derived flow is available during training and inference; it conditions the model rather than directly supervising it.Source: §3.2 · p. 5

3.3 Video Geometry Corrector

• VGC denoises toward the geometry-consistent target while conditioned on the distorted video, clip text prompt, and optical flow maps.Video and flow conditions enter through channel concatenation, while the text prompt is provided through cross-attention.Source: §3.3 · p. 5

4.1 Experimental Setup

• VG-Pairs training uses trajectory-adaptive keyframes, with multiple generated subsegments stitched together for clips with complex camera motion.The implementation constructs data from DL3DV and SpatialVID and uses optical flow as an additional conditioning signal.Source: §4.1 · p. 6
• VGC is evaluated with pose-specified motion on 100 randomly sampled RealEstate10K test videos, and with prompt-induced motion on 200 randomly sampled prompts from the held-out DL3DV test set.For prompt-induced motion, generated videos are independently reconstructed using Depth Anything 3 and evaluated using PSNR, SSIM, LPIPS, MSE, and 3D consistency score (3DCS).Source: §4.1 · p. 6

4.2 Experimental Results

• VGC improves PSNR, SSIM, and RVE over the DFoT baseline on RealEstate10K, and further improves Geometry Forcing and Geometry Forcing + REPA outputs.The comparison indicates that VGC can be applied after geometry-aware generators as well as directly to the baseline.Source: Table 1 · p. 6, §4.2 · p. 6
• Qualitative examples show VGC correcting lighting and structural errors, preserving scene appearance, and recovering plausible camera paths and dynamic motion continuity.Examples include corrected streetlight geometry and restored human identity and motion continuity.Source: Figure 4 · p. 6, Figure 5 · p. 7, §4.3 · p. 8
• Across three CogVideoX generators in text-to-video and image-to-video settings, VGC improves PSNR, SSIM, and MSE over the corresponding baselines and further improves VideoGPA.These results span prompt-driven camera motion and support transfer across generation settings.Source: Table 2 · p. 7, §4.2 · p. 7
• In the RealEstate10K ablation, optical flow improves PSNR, SSIM, and RVE, while the paper attributes VGC’s main gains to paired supervision.The reported flow gains are modest for image quality while long-range geometric consistency remains similar.Source: Table 1 · p. 6, §4.2 · p. 7

4.3 Extension to General Video Refinement

• The larger 14B model improves geometry in dynamic videos, unlike the 1.3B version, and also converts 480p inputs to 720p outputs through super-resolution.Both model sizes correct distortions in static scenes; the 14B model further improves moving humans and objects.Source: §4.3 · p. 7, Figure 6 · p. 8
• On SpatialVID dynamic videos, the 14B VGC is competitive across perceptual and temporal metrics and particularly strong in dynamic degree, imaging quality, and DOVER.Evaluation covers videos generated by CogVideoX, HunyuanVideo, and Wan, using no-reference quality metrics.Source: Table 3 · p. 8, §4.3 · p. 8

• VGC is a post-hoc video-to-video model that corrects geometric distortions in generated videos under camera motion.It maps geometry-distorted videos to geometry-consistent targets without regenerating videos from scratch.Source: §5 · p. 9
• VG-Pairs pairs real videos with diverse camera motion to synthetic geometry-distorted counterparts, providing training targets for VGC.The paper identifies limited supervision for viewpoint-induced appearance change as a source of geometric failures.Source: §5 · p. 9
• Experiments show improved geometric consistency across source generators, while increased model capacity extends the framework toward general video refinement.Optical flow is included as an auxiliary cue to help distinguish what to correct from what to preserve.Source: §5 · p. 9