EPIC: Epipolar-Consistent 360{\deg} Immersive Stereo Video Generation

1 INTRODUCTION

• Existing generators do not directly produce high-resolution stereoscopic 360° video, while immersive stereo data is scarce and costly to capture.Manual creation and capture also make new experiences expensive and difficult to scale.Source: §1 · p. 1
• The pipeline adapts existing video diffusion models to generate text- or image-conditioned stereo pairs without large-scale training data.It uses zero-shot stereo adaptation to extend generation to high-resolution stereoscopic 360° video.Source: §1 · p. 2
• PEGS measures stereo consistency on the 360° sphere, accounting for spherical epipolar geometry while remaining pose-free and robust to independently moving objects.Its rankings form preferred and dispreferred pairs used to optimize the generator with direct preference optimization, reducing artifacts without ground-truth panoramic stereo video.Source: §1 · p. 2
• A stereoscopic panoramic upsampling stage produces 4K output, and baseline adjustment controls disparity for headset viewing.The paper presents these components as a scalable path to high-resolution stereoscopic 360° generation.Source: §1 · p. 2

2 RELATED WORKS

• Omnidirectional stereo rigs synthesize views from a rotating left-right camera arm, but calibration demands, seam artifacts, parallax errors, and fixed baselines constrain capture.The fixed baseline prevents retargeting captured content for a different display or viewer.Source: §2 · p. 2
• Prior video generators cover either pinhole-camera stereo or monocular panoramic video, while the paper addresses their combination with rigid geometric consistency.Table 1 contrasts panoramic and stereo video-generation work; prior work covers one axis at a time, and geometry enters as depth conditioning rather than an explicit epipolar constraint.Source: §2 · p. 2, Table 1 · p. 2
• Recent post-training work uses preference optimization to recover geometric structure that pretraining alone does not supply.Related approaches optimize multi-view consistency [46] or epipolar error [20]; learned aesthetic or vision-language rewards can leave epipolar error essentially unchanged [20].Source: §2 · pp. 2–3
• Direct Preference Optimization [35] uses supervised preference-pair loss without differentiating the reward, unlike rollout-heavy policy-gradient approaches.The related-work discussion notes that full video denoising rollouts make those policy-gradient updates prohibitive.Source: §2 · pp. 2–3
• The closest cited epipolar-reward work [20] measures across time, whereas this paper scores simultaneous stereo views with one essential matrix despite object motion.The passage identifies the temporal method’s single fundamental matrix as valid only for static scenes under camera motion.Source: §2 · p. 3

3.1 Epipolar Constraint

• The epipolar plane constrains the right viewing direction; on panoramas it intersects the sphere as a great circle, rather than a pinhole straight line.For a pinhole camera, the plane cuts the image in a straight line that rectification turns into a horizontal row.Source: §3.1 · p. 3

3.2 Tangent Sampson Error

• Tangent Sampson error measures reprojection distance from epipolar consistency using unit viewing directions and projection-Jacobian pseudoinverses, reducing to classical Sampson error for pinhole cameras.For the equirectangular projection, the Jacobian is closed-form, and the Jacobian terms can be precomputed because they depend only on measured points.Source: §3.2 · p. 3

3.3 Flow Matching for Video Generation

• Rectified flow trains a transformer to predict velocity along a linear noise-to-clean latent path; inference integrates the learned field from noise at t = 1 to clean latents at t = 0.The per-sample velocity error later supplies an objective for steering toward stereo-consistent outputs while keeping the generator close to its pretrained model.Source: §3.3 · pp. 3–4

4 METHOD

• The three-stage pipeline first creates stereo views, then refines geometry through PEGS-guided preference optimization, and finally adapts the output for headset playback.The method section presents these stages as the pipeline for panoramic stereo generation and epipolar refinement.Source: §4 · p. 4, Figure 5 · p. 4, Figure 7 · p. 5

4.1 Stage 1: Training-free Panoramic Stereo

• Depth-guided spherical warping derives the right view from the generated left view; the video diffusion model inpaints only occluded regions, preserving warped content elsewhere.The method uses estimated video depth, smoothed temporally, to guide the warp.Source: §4.1 · p. 4, Figure 5 · p. 4

4.2 Panoramic Epipolar Geometry Score

• PEGS estimates geometric disagreement from confident, tracked correspondences across stereo and temporal view pairs, while validity checks exclude degenerate pairs instead of assigning them zero error.The score uses an angular epipolar violation; tested pairs include same-time left-right views, same-eye views across time, and cross-eye views across time.Source: §4.2 · p. 5

4.3 Stage 2: Preference Optimization for Stereo

• Direct Preference Optimization trains on PEGS-ranked pairs: the lowest-scoring sample is preferred, avoiding reward differentiation and a fresh inference pass for reward computation.The method samples multiple videos for each conditioning and selects the lowest- and highest-scoring samples as the preferred and dispreferred pair.Source: §4.3 · p. 5

4.4 Stage 3: Viewing Refinement

• For omnidirectional stereo, the method estimates left-view depth, scales it by triangulating against the right view, and re-renders columns from eye positions rotated around a small circle.This refinement addresses the limitation that fixed eye positions produce a pair correct only for one viewing direction.Source: §4.4 · pp. 5–6
• The 4K upsampling stage splits each view into cube faces, super-resolves corresponding left-right face pairs together, then reassembles them after omnidirectional projection.Jointly processing the eye pairs keeps their detail consistent; the method projects first and super-resolves last.Source: §4.4 · p. 6

5.1 Dataset and Implementation

• The left view uses a frozen Wan2.1-1.3B backbone and PanoWan equirectangular LoRA; the right view is synthesized by depth-guided backward warping.Evaluation uses 500 videos per stage; preference pairs require at least 5 mrad PEGS difference, and training stops early as policy divergence accelerates.Source: §5.1 · p. 6, Figure 9 · p. 6

5.2 Metrics

• PEGS returns injected frame misalignment with slope 1.000, whereas MEt3R rises then falls across eight sweeps and cannot express a calibrated angular error.Under object motion, PEGS excludes moving tracks; Figure 11 reports 1.10 versus 2.54 mrad when those tracks are retained.Source: §5.2 · p. 6, Figure 11 · p. 7

5.3 Geometric Consistency

• Stereo consistency improves under PEGS (3.121 → 2.640 mrad) and under MEt3R (0.096 → 0.081).The table evaluates two generation sets of 500 videos each; PEGS and MEt3R use different scales, so only values within each metric column are comparable.Source: §5.3 · p. 6, Table 3 · p. 7
• The temporal axis responds to scene motion as well as consistency, and PEGS discards non-rigid tracks before scoring.This makes temporal scores sensitive to which moving content is excluded from the rigid two-view comparison.Source: §5.3 · p. 6

5.4 Comparing against State of the Art

• Three times fewer correspondences fall far off the epipolar curve with our method, which is better on 46 of 50 clips; it is also better on 9 of 10 OmniRoam clips.The DissolveStereo comparison uses 50 clips sharing prompt, seed, and left eye, counting ≥20 mrad as inconsistent; the OmniRoam comparison uses 10 clips sharing the same left eye.Source: §5.4 · p. 7, Table 4 · p. 7, Table 5 · p. 7

5.5 Analysis

• ODS conversion reduces vertical shift from 4.73 px for the rigid pair to 0.04 px, against a ground truth of zero.The rigid pair’s largest shift is near λ = ±90°; the converted pair stays near zero along the measured scanline.Source: §5.5 · p. 7

5.6 Ablations

• With PEGS as the DPO reward, the median improves from 3.10 to 2.64 mrad and the top saturated bin falls from 35 clips to 11; MEt3R’s median moves from 0.053 to 0.081.The authors attribute this difference to PEGS ranking measured epipolar violation, while MEt3R’s feature-space score varies with scene content as well as geometry.Source: §5.6 · p. 7

• The pipeline generates 360° stereo video from a frozen monocular panoramic generator without stereo panoramic training data.A spherical depth warp forms the right eye, with the model filling regions the warp cannot.Source: §6 · p. 7
• PEGS serves as the DPO preference signal and removes gross geometric failures rather than shifting the typical clip.A final conversion stage prepares the pair for omnidirectional stereo headset playback.Source: §6 · p. 7
• PEGS assumes a rigid scene and scores moving content only by discarding it; joint stereo-temporal consistency does not yet improve.The paper identifies this combined consistency axis as its hardest one.Source: §6 · p. 7