PaperlayerView exact revision on arXiv

Source-linked AI summary

OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphs

You-Zhe Xie, Ting-Wei Chou, Yu-Hsuan Li, Kaipeng Zhang, Zhixiang Wang, Yu-Lun Liu

arXiv:2610.12461v1cs.CVcs.GR

TL;DR

Existing 3D cinemagraph methods are limited in viewpoint range or motion type, and fitting generated videos faces loop and multi-view consistency challenges. OuroWorld combines video generation with a periodic 4D representation; it outperforms baselines on its evaluation and is preferred in user-study comparisons.

  • Problem

    Prior 3D cinemagraphs are limited by pseudo-3D viewpoint ranges or fluid-like motion and require masks; generated-video fitting also faces approximate loops and inconsistent views.

    Source: §1 · p. 2

  • Method

    OuroWorld uses a vision-language model to guide reference-video generation, extends the video to multiple views, and fits periodic 4DGS with Fourier deformation and a reference-anchored Grounded Drift Field.

    Source: §1 · p. 2

  • Results

    70.8%–99.0% of user-study comparisons preferred OuroWorld, which outperformed all baselines on the proposed ground-truth-free evaluation.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The resulting scenes exhibit diverse motion and illumination change beyond Eulerian flow, loop exactly by construction, and remain sharp despite inconsistent generated views.

    Source: §5 · p. 10

    Dynamics are bounded by the video-generation prior, while completing all views from one reference video leads to greater disocclusions and inconsistencies at wider viewpoint ranges.

    Source: §5 · p. 10

Abstract

from arXiv · show

Recent 3D world models generate photorealistic, explorable scenes that remain frozen in time. OuroWorld is a mask-free framework that turns any static 3D Gaussian Splatting scene into a 3D cinemagraph: a dynamic scene with vivid, diverse motion looping seamlessly from any viewpoint. A vision-language model infers plausible dynamics and guides a video model to synthesize a reference video, which we lift and complete into multi-view videos. To learn from this imperfect supervision, we propose Inconsistency-Robust Periodic 4DGS: a Fourier-series deformation field guarantees looping by construction, while a Grounded Drift Field anchored at the reference view absorbs cross-view inconsistency. Unlike prior Eulerian methods limited to fluid-like motion, we capture general deformation, object motion, and illumination change. We introduce a ground-truth-free evaluation covering vividness, naturalness, loop seam coherence, and scene quality. On 39 reconstructed and generated scenes, OuroWorld outperforms all baselines and wins 70.8%-99.0% of user-study comparisons. Project page: https://ouroworld.userwei.com

1 Introduction

OuroWorld targets frozen 3D worlds and limitations in prior cinemagraph methods, including masks, restricted motion, and inconsistent generated views. It uses video priors and periodic 4DGS to create looping scenes, outperforming baselines in its evaluation and user study.

1 Introduction

  • OuroWorld targets frozen 3D worlds and prior cinemagraph limits: pseudo-3D viewpoint ranges, fluid-like motion, and required motion masks.Generated videos add two fitting challenges: approximate periodicity leaves seams, while inconsistent multi-view videos can blur scenes and produce erratic motion.Source: §1 · p. 2
  • A vision-language model infers plausible dynamics to guide video generation, and the resulting reference video is extended to multi-view videos for fitting.The framework converts static 3D Gaussian Splatting scenes without motion masks.Source: §1 · p. 2
  • Inconsistency-Robust Periodic 4DGS uses Fourier-series deformation for looping by construction and a reference-view-anchored Grounded Drift Field to absorb cross-view inconsistency.The contributions describe general motion and illumination change beyond Eulerian methods.Source: §1 · p. 2
  • 70.8%–99.0% of user-study comparisons preferred OuroWorld, which also outperformed all baselines under the proposed evaluation.The evaluation is ground-truth-free and covers vividness, naturalness, loop seam coherence, and scene quality.Source: §1 · pp. 2–3

2 Related Work

Related work spans 2D cinemagraphs, static scene representations, dynamic 3D methods, and scene animation. OuroWorld distinguishes itself by producing free-viewpoint 3D motion with representation-enforced looping and without masks.

2 Related Work

  • Earlier cinemagraph methods animate 2D image planes, while OuroWorld produces a 3D scene renderable from arbitrary viewpoints with loop closure enforced by its representation.Prior Eulerian approaches are confined to fluid-like motion; the paper adopts first-and-last-frame conditioning on the same photograph to obtain a reference loop [72].Source: §2 · p. 3
  • Static-scene work ranges from shallow proxies with limited parallax to complete NeRF or 3DGS reconstructions, which also underpin generated explorable worlds.The cited scene-generation approaches include progressive inpainting and lifting [21], diffusion refinement, and large-scale world models [101] [41] [15] [94].Source: §2 · p. 3
  • Dynamic 3D representations add time through deformation, scene flow, space-time grids, per-frame optimization, control points, native 4D primitives, or temporal bases.OuroWorld restricts deformation to a Fourier series for periodic motion and uses a reference-view-anchored, view-conditioned Grounded Drift Field to absorb generative inconsistency.Source: §2 · p. 3
  • Prior 3D cinemagraphs use layered Eulerian flow, asynchronous multi-view captures, or moving Gaussians; scene animation instead borrows motion from video, physics, or language-model-written fields.These methods rely on masks or do not target seamless loops, whereas OuroWorld is mask-free and guarantees an endless loop from any viewpoint.Source: §2 · p. 3

3 Method

OuroWorld turns static 3DGS scenes into looping 3D cinemagraphs through reference-video generation, multi-view video synthesis, and Inconsistency-Robust Periodic 4DGS. Its optimization combines periodic deformation with view-specific drift handling and scene-view-consistent supervision.

3 Method

  • OuroWorld uses a canonical 3DGS, a Periodic Deformation Field, and a Grounded Drift Field to fit generated multi-view supervision in a three-stage framework.The stages generate a looping reference video, extend it to multi-view videos, and optimize the 4DGS representation.Source: §3 · p. 3, §3.3 · p. 4

3.1 Looping Video Generation

  • A VLM infers plausible cyclic motion from a rendered reference image, then guides a video model to generate and filter an approximately looping 10-second reference video.The reference image is used as both the first and last frame.Source: §3.1 · p. 4

3.2 Multi-view Video Generation

  • VGGT-Ω lifts the reference video into a dynamic point cloud, which is rendered from 20 nearby viewpoints and inpainted to produce multi-view videos.The viewpoints are sampled uniformly within ±20° of horizontal rotation around a pivot; static 3DGS renders improve depth estimation.Source: §3.2 · p. 4

3.3.1 Periodic Deformation Field

  • A Fourier series parameterizes each Gaussian’s temporal variation, making deformation T-periodic by construction and capable of approximating arbitrary smooth periodic motion.The resulting representation extends beyond fluid-like motion, and the periodicity guarantees looping regardless of the optimization outcome.Source: §3.3.1 · p. 5

3.3.2 Grounded Drift Field

  • The Grounded Drift Field predicts view-specific, time-varying offsets; grounding the reference view forces the shared deformation to reproduce its dynamics while drift absorbs other views’ residual deviations.At inference, drift is discarded; unlike ICP-based drift estimation, the learned offsets cover position, rotation, scale, and color.Source: §3.3.2 · p. 5

3.4 Scene-View Consistent Optimization

  • Scene-view-consistent optimization applies an ℓ1 loss to the reference video and time-zero input renders, then supervises other frames with LPIPS against diffusion-refined images.The total objective combines the two losses as L = λ1L1 + λpLp.Source: §3.4 · pp. 5–6

4 Experiments

Experiments compare OuroWorld with prior methods on 39 static 3DGS scenes using a ground-truth-free metric suite and a user study. The reported results favor OuroWorld across key metrics and user preferences.

4.1 Dataset; 4.2 Baselines

  • The 39-scene benchmark combines nine Mip-NeRF 360 scenes with 10 each from HY-World 2.0, Marble, and Lyra 2.0, and compares OuroWorld with four methods.Scenes include indoor rooms, landscapes, architecture, and stylized environments.Source: §4.1 · p. 6, §4 · p. 6, §4.2 · p. 6

4.3 Evaluation Protocol; 4.4 Evaluation Metrics

  • The ground-truth-free evaluation measures vividness, naturalness, loop seam coherence, and scene quality because prior ground-truth-based evaluations use limited fluid or object-level data.The methods are rendered under static and orbit camera trajectories.Source: §4.4 · p. 6, §4.3 · p. 6

4.6 Quantitative Results

  • OuroWorld achieves the best Vividness, Naturalness, MALF, and Aesthetic Quality under both camera settings, while combining high Seam SSIM with the highest MALF.Near-static methods can attain high Seam SSIM but have near-zero MALF.Source: §4.6 · p. 7, Table 2 · p. 7

4.7 Qualitative Results

  • Under the orbit camera, OuroWorld shows vivid, view-consistent dynamics, while 3D Cinemagraphy and 3D-MOM show disocclusion holes and color artifacts and two other baselines remain nearly static.Examples include changing illumination, rippling water, drifting fog, and swaying curtains.Source: Figure 5 · p. 8, §4.7 · p. 8

4.9 Ablation Study

  • Removing the Grounded Drift Field lowers detail sharpness and consistency across cameras; without drift, clouds fade, a galaxy blurs, and ground rocks vanish over time.The full model preserves these details while maintaining vivid, coherent motion.Source: §4.9 · pp. 8–9, Figure 7 · p. 9

4.8 User Study

  • Participants preferred OuroWorld in 70.8%–99.0% of comparisons across baselines and four criteria, with the largest margins in Vividness.The two-alternative forced-choice study included 30 participants.Source: §4.8 · p. 8, Figure 6 · p. 9

4.9 Ablation Study

  • Applying drift to the reference view lets it absorb motion, lowering static-camera Vividness from 0.0423 to 0.0392; grounding requires the periodic field to reproduce the motion.Source: §4.9 · p. 9
  • Naive optimization across generated views yields the lowest Subject Consistency and a 22% orbit-camera VoL drop; doubling the Fourier period lowers static-camera Seam SSIM from 0.9965 to 0.9783.The doubled-period variant retains nearly unchanged MALF, concentrating degradation at the loop boundary.Source: §4.9 · pp. 9–10

5 Conclusion and Limitation

OuroWorld brings static 3DGS scenes to life as seamlessly looping 3D cinemagraphs, with diverse motion and illumination change. Its generated-view supervision has limitations, especially at wider viewpoint ranges.

5 Conclusion and Limitation

  • OuroWorld’s scenes exhibit diverse motion and illumination change beyond Eulerian flow, loop exactly by construction, and remain sharp despite inconsistent generated views.Experiments on reconstructed and generated scenes show consistent gains over prior 3D cinemagraph methods.Source: §5 · p. 10
  • Dynamics remain bounded by the video-generation prior, while completing views from one reference video leads to more disocclusions and inconsistencies at wider viewpoint ranges.Generating multiple reference videos from distinct viewpoints is proposed as a promising remedy.Source: §5 · p. 10

Appendix Overview

The appendix documents the evaluation-scene collection, pipeline settings, optimization, baselines, metrics, and user-study protocol, with rendered method videos in the supplement.

Appendix Overview

  • Appendix A.1 describes how the 39 evaluation scenes are collected from Mip-NeRF 360, HY-World 2.0, Marble, and Lyra 2.0.Source: Appendix Overview · p. 11
  • Appendix A.2 details the VLM prompt, video-generation and lifting settings, and Inconsistency-Robust Periodic 4DGS hyperparameters.Source: Appendix Overview · p. 11
  • Appendix A.3 formulates scene-view consistent optimization with diffusion-based refinement.Source: Appendix Overview · p. 11
  • Appendices A.4–A.6 cover baselines and their inputs and loop periods, metric details, and the user-study protocol and interface.Source: Appendix Overview · p. 11
  • Rendered videos of all methods are included in the supplementary material.Source: Appendix Overview · p. 11

A.1 Details on Dataset Collection

The benchmark combines 39 static 3DGS scenes from four sources, selected and prepared to support evaluation across varied environments and camera trajectories. The appendix details scene reconstruction, keyframe selection, initialization, and optimization.

A.1 Details on Dataset Collection

  • The benchmark contains 39 scenes: nine reconstructed from Mip-NeRF 360 and ten each generated by HY-World 2.0, Marble, and Lyra 2.0.Each scene has a reference camera pose and image, around which evaluation trajectories are defined.Source: §A.1 · p. 11
  • Generated scenes were selected for natural dynamic content, varied environment types, and a clean region supporting a ±20° orbit; reference poses were manually chosen within that region.Source: §A.1 · p. 11
  • Mip-NeRF 360 contributes all nine scenes, while HY-World 2.0 and Marble scenes are downloaded as Gaussian-splat files and used without regeneration or re-optimization.Lyra 2.0 scenes are generated locally from repository sample images and captions; its exploration trajectory is modified to use a dolly followed by a wide orbit, yielding 480 frames per scene.Source: §A.1 · pp. 11–12
  • On long, wide-baseline videos, the released feed-forward reconstructor produces blurry scenes with many floaters, so reconstruction combines foundation-model poses and initial geometry with per-scene 3DGS optimization.Source: §A.1 · p. 12
  • The reconstruction pipeline selects 64 keyframes per video, refining optical-flow-based picks using VGGT camera paths and equal 3D arc-length spacing.Source: §A.1 · p. 12
  • VGGT supplies camera parameters and confidence-scored depth for initialization; low-confidence and depth-boundary pixels are discarded before point-cloud export to COLMAP.The point cloud is voxel-downsampled at 0.006 in VGGT normalized scene units.Source: §A.1 · p. 12
  • Standard 3DGS optimization runs for 30,000 iterations with an ℓ1 + 0.2·D-SSIM loss, reaching mean held-out novel-view PSNR of 22.1 dB and SSIM of 0.71.Every eighth keyframe is held out for evaluation; adaptive density control stops at 2,000 iterations.Source: §A.1 · p. 12

A.2 Details on Our Pipeline

Input scenes are standardized to spherical-harmonics degree 3 so their canonical colors share one parameterization.

A.2 Details on Our Pipeline

  • Scenes with SH degree 0 are converted to degree 3 by zero-initializing missing higher-order coefficients, matching the canonical-color parameterization across sources.Source: §A.2 · p. 12

A.2.1 Looping Video Generation

OuroWorld uses a dynamics prompt to guide video generation, with the reference image anchoring both ends of the generated clip. The prompt is also reused for baseline and KVD-reference video generation.

A.2.1 Looping Video Generation

  • GPT-5.5 uses the reference image to write a dynamics prompt that requests visible motion, multiple moving elements where appropriate, and no new content or objects crossing the camera.The prompt conditions video generation and is reused as text input for baselines.Source: §A.2.1 · p. 12
  • A second video-generation prompt, produced with the same GPT prompt, conditions the MiniMax H3 videos used as KVD references.The dynamics prompt is also supplied to the baselines.Source: §A.2.1 · p. 12
  • Seedance 2.0 receives the reference image as both the first and last frame, encouraging the generated video to return to its initial state.Videos are generated at 720p, 24 FPS, for 10 seconds without audio.Source: §A.2.1 · p. 12

A.2.2 Dynamic Point Cloud Lifting

OuroWorld lifts the reference video into a dynamic point cloud using VGGT-Ω and aligns that reconstruction to the input 3DGS coordinate frame. The inputs combine static orbit views and temporal frames from the reference video.

A.2.2 Dynamic Point Cloud Lifting

  • VGGT-Ω reconstructs the dynamic point cloud in one forward pass from 45 images: 21 static multi-view renders and 24 temporal reference-video frames.The static views follow a horizontal orbit with azimuths from −20° to 20° at 2° intervals; the central view is the reference view.Source: §A.2.2 · p. 12
  • The 25 sampled temporal frames include the reference image as the first frame, so that frame is dropped and the remaining 24 are assigned to the reference view.The reference image matches the static render at zero azimuth.Source: §A.2.2 · p. 12
  • All images share a center crop and resize to a longer side of 512 pixels, while predicted camera poses are discarded and the point cloud is affine-aligned to the input 3DGS.The 10% least-confident points of each frame are removed; the input 3DGS cameras are used instead.Source: §A.2.2 · p. 13

A.2.3 Multi-View Video Generation

The appendix details how OuroWorld generates and fits multi-view supervision, evaluates motion and loop coherence, and conducts baseline and user-study comparisons. Its ablations show that grounding and scene-view consistent optimization affect distinct motion and quality outcomes.

A.2.3 Multi-View Video Generation

  • Twenty training cameras orbit the selected pivot within ±20° at the reference camera’s radius, and TrajectoryCrafter completes the rendered dynamic-point-cloud videos using shared initial noise.The evaluation orbit uses the same pivot at 0.95 times the training radius.Source: §A.2.3 · p. 13

A.2.4 Inconsistency-Robust Periodic 4DGS

  • The Periodic Deformation Field uses a Fourier-feature triplane and separate output heads, while the Grounded Drift Field conditions on time and view embeddings with a spatial feature shared across non-reference views.The shared feature is motivated by correlated inconsistencies across independently inpainted views.Source: §A.2.4 · p. 13

A.3 Details on Scene-View Consistent Optimization

  • Consistent reference-video and start-time renders receive pixel-level supervision, while other generated frames guide perceptual refinement rather than serving as pixel targets.The latent refinement blends the current rendering toward the generated frame with a decreasing influence over five denoising steps; optimization begins with L1 alone before using the full objective.Source: §A.3 · pp. 14–15

A.4.1 Methods

  • The evaluated baselines use different motion constructions: Gaussians-to-Life drives Gaussians with lifted point flow, while the other described methods use Eulerian motion with layered-depth or Gaussian representations.Gaussians-to-Life does not target looping; 3D-MOM’s optimized 4D representation is not guaranteed to loop.Source: §A.4.1 · p. 15

A.4.2 Input Alignment

  • Baseline inputs are aligned around the same scene, view, and motion description, with masks derived from the prompt and verified by a human before use.Gaussians-to-Life alone requires a 3D mask, lifted by selecting in-mask Gaussians within a depth interval.Source: §A.4.2 · p. 15

A.4.3 Native Periods; A.5.2 Naturalness

  • Methods retain their native periods for the 20-second evaluation videos, while naturalness is measured with KVD against MiniMax H3 reference videos.Static-camera and orbit-camera KVD scores are comparable within, but not across, camera settings.Source: §A.4.3 · p. 16, §A.5.2 · p. 17

A.5.1 Vividness

  • Removing grounding reduces static-camera Motion Variation from 0.0413 to 0.0324, while removing scene-view consistent optimization raises vividness components amid inpainting flicker and gives the lowest consistency and sharpness.OuroWorld also leads in Illumination Variation; the authors attribute this to illumination changes that Eulerian baselines cannot express.Source: §A.5.1 · p. 16, Table A.4 · p. 17

A.5.3 Loop Seam Coherence; A.6 Details on User Study

  • MALF measures phase-aligned return across cycles and cannot be improved by suppressing motion, whereas Seam SSIM measures the boundary jump but can be high for nearly static videos.The user study compares paired videos on vividness, naturalness, loop seam coherence, and visual quality; 30 participants judged 156 pairs across 39 scenes and four baselines.Source: §A.5.3 · p. 17, §A.6 · pp. 17–18
Loading 2610.12461v1…