Source-linked AI summary
FlashWorld: High-quality 3D Scene Generation within Seconds
Xinyang Li, Tengfei Wang, Zixiao Gu, Shengchuan Zhang, Chunchao Guo, Liujuan Cao
TL;DR
Full 3D scene generation must reconcile visual quality, 3D consistency, efficiency, and limited scene data. FlashWorld combines dual-mode pre-training with cross-mode post-training distillation and out-of-distribution co-training. It reports state-of-the-art performance on multiple tasks with significant inference-speed advantages, while retaining limitations in scene coverage and fine-grained content.
Problem
Full 3D scene generation is limited by scarce high-quality 3D scene data, complex real-world scenes, and trade-offs between MV-oriented quality and 3D-oriented consistency.
Method
FlashWorld pre-trains a dual-mode multi-view diffusion model, then distills the MV-oriented teacher into a 3D-oriented Gaussian generator while co-training on images and text prompts.
Results
FlashWorld achieves state-of-the-art performance on multiple tasks while offering significant inference-speed advantages and improved visual fidelity with 3D consistency.
Takeaways & Limitations
Cross-mode distillation combines high visual fidelity from MV-oriented generation with the 3D consistency of direct Gaussian generation.
Takeaways & Limitations
Scene diversity and scale remain constrained by existing dataset coverage, and the model struggles with fine-grained geometry, mirror reflections, and articulated objects.
Abstract
from arXiv · showhide
We propose FlashWorld, a generative model that produces 3D scenes from a single image or text prompt in seconds, 10~100$\times$ faster than previous works while possessing superior rendering quality. Our approach shifts from the conventional multi-view-oriented (MV-oriented) paradigm, which generates multi-view images for subsequent 3D reconstruction, to a 3D-oriented approach where the model directly produces 3D Gaussian representations during multi-view generation. While ensuring 3D consistency, 3D-oriented method typically suffers poor visual quality. FlashWorld includes a dual-mode pre-training phase followed by a cross-mode post-training phase, effectively integrating the strengths of both paradigms. Specifically, leveraging the prior from a video diffusion model, we first pre-train a dual-mode multi-view diffusion model, which jointly supports MV-oriented and 3D-oriented generation modes. To bridge the quality gap in 3D-oriented generation, we further propose a cross-mode post-training distillation by matching distribution from consistent 3D-oriented mode to high-quality MV-oriented mode. This not only enhances visual quality while maintaining 3D consistency, but also reduces the required denoising steps for inference. Also, we propose a strategy to leverage massive single-view images and text prompts during this process to enhance the model's generalization to out-of-distribution inputs. Extensive experiments demonstrate the superiority and efficiency of our method.
1 INTRODUCTION
FlashWorld addresses the quality and efficiency challenges of full 3D scene generation by combining MV-oriented visual fidelity with 3D-oriented consistency. Its dual-mode pre-training, cross-mode distillation, and out-of-distribution co-training target faster, more generalizable generation.
- Full 3D scene generation remains difficult because high-quality 3D scene data are scarce and real-world scenes have exponential modeling complexity.
- MV-oriented pipelines generate multiple views before reconstruction but lack explicit 3D constraints, causing geometric and semantic inconsistencies.
- 3D-oriented pipelines directly generate scenes without reconstruction, but often produce blurry artifacts and require refinement that reduces efficiency.
- FlashWorld introduces dual-mode pre-training based on a video diffusion model, supporting both MV-oriented and 3D-oriented generation modes.
- Cross-mode post-training uses the MV-oriented mode as teacher for visual quality and the 3D-oriented mode as student for 3D consistency.
- Massive unlabeled images and text prompts with simulated camera trajectories are used during post-training to improve generalization to diverse inputs.
2 PRELIMINARY
Diffusion models generate data by reversing a noise process, while distribution matching distillation transfers a slow teacher’s generation behavior to a fast few-step student. The approach relies on score estimates for real and student distributions.
- Diffusion models transform standard Gaussian noise into samples from a target data distribution through progressive denoising.
- Denoising networks may predict the original data, noise, or v, with predictions convertible to a denoised estimate and distribution score.
- Distribution matching distillation minimizes approximate KL divergence between smoothed real-data and student-output distributions across sampled timesteps and noise inputs.
- DMD freezes the pre-trained real-distribution diffusion model as teacher while dynamically updating a fake-score model during generator training.
3 METHOD
FlashWorld combines dual-mode pre-training with cross-mode post-training to generate 3D scenes directly as Gaussian representations while improving visual quality, efficiency, and robustness to diverse inputs.
- 3.1 DUAL-MODE PRE-TRAINING: Dual-mode pre-training builds one multi-view diffusion model that supports both image-oriented generation and direct 3D Gaussian generation.The model is trained with multi-view images, camera parameters, and conditioning such as text prompts or single-view images.
- 3.1 DUAL-MODE PRE-TRAINING: A video diffusion prior supports faster convergence, larger view counts, and higher output resolution than image-diffusion initialization.The described configuration supports 24 views and 480P outputs.
- 3.2 CROSS-MODE POST-TRAINING: The framework distills high-quality MV-oriented generation into a 3D-oriented student, combining visual quality with 3D consistency and faster inference.The MV-oriented mode serves as teacher, while the 3D-oriented mode provides the student initialization and consistency-preserving representation.
- 3.2 CROSS-MODE POST-TRAINING: Inference alternates 3D-oriented denoising with noise injection, using rendering at each step to maintain 3D consistency with few denoising steps.The schedule typically uses a small number of timesteps, such as N = 4.
- 3.3 OUT-OF-DISTRIBUTION DATA CO-TRAINING: Post-training additionally co-trains on image or text conditions with random camera trajectories to broaden inputs beyond limited multi-view data.The strategy targets diverse scenes and improves the 3DGS decoder’s ability to process out-of-distribution inputs.
4 EXPERIMENTS
FlashWorld is evaluated on image-to-3D, text-to-3D, and WorldScore tasks, with strong visual quality, text alignment, consistency, and inference efficiency. Ablations show that cross-mode consistency and out-of-distribution data improve distinct aspects of generation.
- 4.2 COMPARISON ON TEXT-TO-3D SCENE GENERATION: FlashWorld produces fine-grained objects and realistic backgrounds while preserving semantic consistency under text-to-3D generation.Competing methods exhibit blurry artifacts, incorrect geometries, or difficulty reproducing details such as fur, floors, and grass.
- 4.2 COMPARISON ON TEXT-TO-3D SCENE GENERATION: FlashWorld attains the highest CLIP Score on two text-to-3D subsets and retains its speed advantage at higher resolution and with more frames.The unified model handles image-to-3D and text-to-3D without separate training processes.
- 4.3 COMPARISON ON WORLDSCORE BENCHMARK: FlashWorld achieves the highest average WorldScore and fastest inference among compared approaches, while ranking best in Style Consistency.It ranks second in Photometric Consistency, Object Control, and Subjective Quality, but lower in 3D Consistency and Content Alignment.
- 4.4 ABLATION STUDY: Ablations show that MV-oriented variants suffer noisy reconstruction, 3D-oriented diffusion produces blurry results, and removing cross-mode consistency causes floating and duplicated artifacts.Removing out-of-distribution data increases semantic misalignment and lowers quantitative text-alignment metrics.
5 CONCLUSION
FlashWorld combines dual-mode pre-training, cross-mode post-training, and out-of-distribution data co-training for efficient 3D scene generation. The paper reports state-of-the-art performance across multiple tasks with substantial inference-speed advantages.
- 5 CONCLUSION: FlashWorld transfers visual fidelity from MV-oriented diffusion to a 3D-oriented model while retaining 3D consistency.Its training uses dual-mode pre-training and cross-mode post-training, with out-of-distribution data co-training to improve generalization.
- 5 CONCLUSION: The method achieves state-of-the-art performance on multiple tasks while offering significant inference-speed advantages.The authors identify autoregressive generation and dynamic 4D scene generation as future directions.
A TRAINING DETAILS
FlashWorld uses a dual-mode multi-view latent diffusion model with specified training, precision, distributed-computing, and dataset configurations. Pre-training uses multi-view data, while post-training additionally incorporates out-of-distribution image-text data.
- Architecture configuration: The model is initialized from WAN2.2-5B-IT2V and uses 24 key frames, 16× spatial latent downsampling, 1024-channel auxiliary features, and a CNN discriminator head.These settings apply to the dual-mode multi-view latent diffusion architecture.
- Pre-training configuration: Pre-training runs for 20,000 steps with a 1,000-step warm-up, learning-rate decay, Adam optimization, and approximately three days of training.The transformer and 3DGS decoder use learning rates of 2 × 10^-6.
- Post-training configuration: Post-training uses four generator timesteps, updates the fake score network four times per generator update, and applies separate generator and discriminator learning rates.The timestep schedule is {1000, 900, 750, 500}; the learning rates are 1 × 10^-6 and 5 × 10^-7.
- Training infrastructure: Both phases use bf16 precision, batch size 64, 64 NVIDIA H20 GPUs, FSDP, activation checkpointing, and flow matching.The MV-oriented mode uses v-prediction following the video diffusion initialization.
- Dataset configuration: Pre-training and post-training use MVImgNet, RealEstate10K, and DL3DV10K multi-view datasets covering object-centric, indoor, and general-purpose scenes.The datasets use resolutions of 480×704 or 704×480 with specified frame-stride ranges.
- Dataset configuration: Post-training adds arbitrary image-text pairs with scene camera trajectories and stylized Echo4O images paired with WildRGBD trajectories.These datasets are used as out-of-distribution data.
B RELATED WORKS
Related work spans iterative reconstruction, multi-view-oriented generation followed by reconstruction, 3D-oriented rendering during denoising, and diffusion-model distillation. These paradigms differ in how they balance consistency, quality, and efficiency.
- Iterative 3D scene generation: Iterative methods generate scenes through repeated view synthesis, depth-conditioned diffusion, or refinement of mesh-based representations.Examples include DiffDreamer, SceneScape, Text2Room, RGBD2, and WonderJourney.
- Multi-view-oriented 3D scene generation: Multi-view-oriented methods first generate novel views or coherent videos and then reconstruct 3D scenes from those frames.CAT3D, DimensionX, and ODIN represent this two-stage pipeline.
- 3D-oriented 3D scene generation: 3D-oriented methods render 3D representations during denoising to generate scenes directly.DMV3D uses triplane NeRF reconstruction and rendering, while Dual3D uses a dual-mode latent diffusion model with neural surface rendering.
- Distillation for diffusion models: Diffusion distillation transfers knowledge from a pretrained teacher to a more compact student to improve efficiency.Denoising Student uses RMSE matching, DMD uses reverse KL-divergence, and DMD2 adds a GAN objective for multi-step generators.
C LIMITATIONS
FlashWorld generates high-fidelity and efficient 3D scenes, but its scene diversity, scale, and fine-grained geometric fidelity remain constrained.
- Existing-dataset coverage constrains the diversity and scale of generated scenes, even when increasing the number of views.The authors identify dataset coverage as a continuing boundary on scene generation.
- FlashWorld currently struggles with fine-grained geometry, mirror reflections, and articulated objects.The paper suggests depth priors and more 3D-aware structural information as possible directions for improvement.
D RGBD RENDERING RESULTS
FlashWorld produces RGBD renderings without explicit depth supervision, indicating that image supervision alone can yield meaningful depth geometry.
- FlashWorld exports depth maps from its generated 3DGS outputs despite using no explicit depth supervision.The paper presents RGBD rendering results in Fig. 8.
- The RGBD results demonstrate that the model learns meaningful depth geometric information solely from image supervision.
- Generated 3DGS representations support RGBD rendering as an output capability.
E MORE RESULTS
Additional results cover diverse scene types and are rendered with generated 3DGS, while video results are deferred to the project page.
- Additional results span object-centric, indoor, outdoor, realistic, and stylized scenes, demonstrating broad generation coverage.
- Video rendering results are available on the project page rather than in this section.
- All images in the additional results are rendered with generated 3DGS.