Single Image to Textured 3D Object Generation in Frequency Domain: From Theory to Pipeline

• Single-view reconstruction is difficult because a single image provides insufficient 3D information.Source: §1 · p. 1
• 2D priors can produce color deviation and view inconsistency, whereas 3D priors improve structural consistency but lack texture details.Source: §1 · pp. 1–2
• Spatial-domain blending of 2D and 3D priors improves structures and textures but still leaves color deviation and the Janus Problem.Source: §1 · p. 2
• The paper asks whether score-distillation gradients and multiple diffusion priors can be integrated in a decoupled frequency-domain optimization framework.Source: §1 · p. 2
• Morpheus3D uses 3D guidance for shape and coarse textures, then high-pass image-prompt 2D guidance to enhance high-frequency details.Source: §1 · pp. 2–3
• The paper analyzes diffusion priors from a frequency perspective and presents a unified hybrid optimization framework for controlling their guidance.Source: §1 · p. 3

• Single-view 3D reconstruction is challenging because learning from one image provides imprecise geometric structures and textures.Source: §2 · p. 3
• Early methods focused on specific object classes or object-centric datasets, while newer feed-forward methods benefit from expanding 3D data.Source: §2 · p. 3
• 3D data remains much scarcer than 2D data, limiting feed-forward models on out-of-domain and in-the-wild single images.Source: §2 · p. 4
• Large pretrained visual models are used to extract implicit 3D information when 3D annotation data is limited.Source: §2 · p. 4
• Pretrained diffusion models have become priors for zero-shot downstream tasks, but text-prompt priors can provide unrealistic guidance because textual semantics are ambiguous.Source: §2 · p. 4

3.1 Diffusion Models

• Diffusion models add noise in a forward process and learn a reverse process that progressively denoises Gaussian noise into clean data.Source: §3.1 · p. 4
• The diffusion objective trains a noise-prediction model by minimizing a weighted or simple squared error between predicted and actual noise.Source: §3.1 · p. 4
• The diffusion score is related to the predicted noise, enabling pretrained diffusion models to serve as priors for single-view 3D reconstruction.Source: §3.1 · p. 4

3.2 Variational Score Distillation

• Score distillation extracts 3D information from diffusion priors to guide the optimization of 3D representations.Source: §3.2 · p. 5
• Variational Score Distillation models distributions of 3D representations and rendered images, then expresses optimization through a weighted KL-divergence objective.Source: §3.2 · p. 5
• VSD solves the distribution-matching objective through gradient flow using predicted noise from the prior and fine-tuned model.Source: §3.2 · p. 5

Analysis of Different Diffusion Priors

• Text-prompt 2D priors provide high-frequency details but can introduce Janus-style view inconsistency because text has high information entropy.The method therefore avoids text-prompt priors for reconstruction.Source: §4.1 · p. 5
• Image-prompt 3D priors provide strong 3D consistency but lack high-frequency details and tend to be over-smoothed.The paper identifies this limitation through power spectral density analysis and reports that VSD does not fully overcome it.Source: §4.1 · p. 5, Figure 3 · p. 5

Hybrid Optimization using Multiple Diffusion Priors with VSD in Frequency Domain

• The unified framework represents rendered and reference images in the frequency domain and applies weighted guidance from multiple diffusion priors.It extends VSD from spatial to frequency-domain optimization, with spatial-domain multi-prior optimization appearing as a special case.Source: §4.2 · pp. 6–7

Analysis of Different Diffusion Priors

• Image-prompt 2D priors use Canny edges from coarse renderings as implicit camera-pose conditions, preserving object consistency while introducing low-frequency color deviation.PFD is used as the image-prompt 2D prior through ControlNet conditioning.Source: §4.1 · pp. 5–6, §4.3.1 · p. 8

Datasets and metrics

• Morpheus3D is evaluated on Realfusion15 and MorpheusObj30 using reference-view and novel-view reconstruction metrics.The evaluation includes PSNR and LPIPS for reference views, and MANIQA and CLIPIQA for novel views.Source: §5.1 · p. 10, §5.2 · p. 11

Quantitative comparison

• +0.195 CLIP-Similarity and +0.157 CLIPIQA are reported over SyncDreamer and TRELLIS, respectively, on Realfusion15.The reported gains correspond to 30.33% semantic-consistency improvement and 26.57% generation-quality improvement.Source: §5.3 · p. 13

Ablation study

• High-pass 2D prior guidance outperforms null-pass and full-pass guidance on novel-view generation quality and semantic consistency.The ablation evaluates MANIQA, CLIPIQA, and CLIP-Similarity, while full-pass guidance introduces heavier color deviation.Source: §5.4 · pp. 14–15
• Fine-tuning the 2D prior overcomes SDS oversmoothing, boosting high-frequency details and improving novel-view generation quality and semantic consistency.Without fine-tuning, high-pass guidance remains close to the Base condition; training-process PSD measurements show increased high-frequency content with fine-tuning.Source: §5.4 · p. 15

Study motivation

• The section raises why VSD cannot be used directly after the preceding experiments, without stating an answer in the supplied passage.No further evidence is provided for the study's findings.Source: §5.5 · p. 16

Computational comparison

• Morpheus3D achieves the highest reported performance among optimization-based methods while consuming the least training time in that comparison.The method avoids textual inversion and uses a coarse-to-fine two-stage pipeline.Source: §5.6 · pp. 17–18
• Feed-forward methods often generate results in minutes, whereas optimization-based methods often require hours.Despite this broader speed difference, Morpheus3D is reported as the most efficient among the optimization-based methods.Source: §5.6 · p. 18

Feed-forward refinement

• Applying Morpheus3D to TRELLIS-generated assets improves generation quality and visibly enhances texture details.The pre-generated textured mesh replaces the standard Instant-NGP NeRF initialization before subsequent optimization.Source: §5.7 · pp. 18–19

Conclusion

• The framework addresses text-prompt 2D prior entropy, image-prompt 2D color deviation, and image-prompt 3D high-frequency deficiency.It uses frequency-domain hybrid optimization to extract complementary advantages from multiple priors.Source: §6 · p. 19
• Morpheus3D is a two-stage pipeline for generating 3D objects from a single unposed image in the wild.The reported scope includes suppression of view inconsistency, low-frequency deviation, and high-frequency lack.Source: §6 · p. 19

Limitation

• The prior weights and frequency-bound filtering rate are handcrafted and fixed, requiring heuristic design.The authors state that these parameters are nevertheless quite robust for generation quality.Source: §6 · p. 19

• The authors strongly recommend watching more visual results.Source: A More Results · p. 19
• The Morpheus3D project webpage is provided as one viewing option.Source: A More Results · p. 19
• The supplementary-material video is provided as another viewing option.Source: A More Results · p. 19

• Parseval’s Theorem states that total signal energy is identical in spatial and frequency domains.Source: §B.1 · p. 19
• The theorem is expressed for a signal x(t) and its spectral representation u(f).Source: §B.1 · p. 19
• The section also states a global-optimum result for VSD for each t > 0.Source: §B.1 · p. 19

• The proof begins from the optimization object and rewrites it into an equivalent form.Source: §B.2 · p. 20
• The objective is transformed through an upper-bound optimization aligned with ELBO and the VSD construction.Source: §B.2 · p. 20
• The weighting function ω(t) is time-dependent, while αt and σt are degradation coefficients.Source: §B.2 · p. 20
• The resulting distribution µ(θ|y) is optimized in the 2-Wasserstein space W2(Θ), yielding a gradient flow.Source: §B.2 · p. 20
• The derivation uses the Fokker–Planck formulation to obtain an intermediate equation before the final gradient-flow expression.Source: §B.2 · p. 20

• The section states a global optimum of VSD in the frequency domain for each t > 0.Source: §B.3 · p. 20
• It introduces a frequency-domain conversion of the diffusion score function from ∇xt log pt(xt).Source: §B.3 · pp. 20–21
• The proof analyzes each frequency component ui using diagonal matrices Λi whose entries consist of 0 and 1.Source: §B.3 · pp. 20–21
• The frequency-domain optimization objective is transformed through a gradient-flow derivation based on the frequency-domain score relation.Source: §B.3 · pp. 21–22
• The proof reparameterizes x0 as y + σϵ with ϵ ∼ N(0, I) and derives the corresponding frequency variables.Source: §B.3 · p. 21
• When Λi = I, the resulting gradient flow is equivalent to the spatial-domain expression.Source: §B.3 · p. 22

B.4 Proof of Proposition 3

• The proof decomposes the weighted optimization objective and its guidance into individual frequency components, following the earlier Proposition 2 derivation.The resulting component-wise gradient flow is assembled into the full objective.Source: §B.4 · p. 22

Implementation assumptions

• The implementation assumes shared diffusion hyper-parameters across priors, while allowing personalized objectives when priors use different settings.The shared-parameter assumption is motivated by current priors inheriting settings from Stable Diffusion.Source: §B.4 · p. 22

Theoretical framework implementation

• The framework’s implementation combines diffusion-prior gradients with frequency-domain transforms and weighted reconstruction losses before updating generator parameters.The pseudocode applies DCT and inverse DCT operations, filters frequency components, and accumulates weighted gradients.Source: §C.1 · pp. 22–23

Pipeline implementation

• The two-stage pipeline first uses a 3D prior for reconstruction, then combines 2D and 3D priors while applying high-pass 2D guidance and updating a LoRA model.The second stage computes DCT-domain high-pass components for the 2D prior and retains the 3D prior gradient.Source: §C.2 · p. 23

Experimental settings

• Experiments use fixed camera and training settings, including unposed-image camera ranges, specified diffusion priors, regularization losses, and reproduction adjustments for competing methods.The reported setup includes elevation and azimuth ranges, fixed field of view, staged time schedules, and method-specific evaluation resolutions or views.Source: D More Details and Hyper-parameters · pp. 23–24