DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders

1. Introduction

• DiffusionBrowser provides consistent previews at denoising steps or blocks, allowing users to stop unpromising generations early without compromising the model’s full capacity.The framework is described as lightweight and having negligible overhead.Source: §1 · p. 2
• The multi-branch decoder predicts RGB and intrinsic channels, which emerge early and provide a richer 3D preview for examining diffusion behavior.The intrinsic channels include albedo, depth, and surface normals.Source: §1 · p. 2
• Preview decoders also let users steer denoising trajectories at branching points using early signals such as layout, motion, and appearance.The paper demonstrates color, depth, and normal steering.Source: §1 · p. 2

2. Related Work

• Prior efficiency approaches include distillation, cascaded diffusion, autoregressive models, mixture-of-experts, and sparse attention, while the paper frames DiffusionBrowser as model-agnostic and fidelity-preserving.The related-work discussion notes that prior approaches can degrade quality or diversity, require full-step inference, or involve complex pipelines or capacity changes.Source: §2 · p. 2
• Earlier studies connect generative-model features to geometry and shading [9] [11] [12], and this paper uses video diffusion features to predict intrinsics and analyze them.The paper identifies preview generation and steering as a tool for analyzing video diffusion features.Source: §2 · p. 2
• Unlike reward-guided tree search that evaluates rewards after rollout, this work provides intermediate previews for user steering.The related-work discussion says this avoids expensive reward evaluation and can complement that approach.Source: §2 · p. 2
• Probing results show intrinsic prediction saturates around timesteps 5–15 of 50 and blocks 10–20 of 30, while RGB prediction improves with depth and timestep.Depth and normals are more easily predicted at earlier stages.Source: Figure 2 · p. 3

3.1. Diffusion Models

• Diffusion models iteratively denoise data, and the section describes simulating continuous dynamics with discrete updates to the learned vector field.Scene intrinsics are described as geometric, shading, and lighting information, including depth, normals, albedo, roughness, and metallicity.Source: §3.1 · p. 3, §3.2 · p. 3

4. Interactive Preview Generation

• The preview objective is to preserve enough appearance and motion for users to anticipate the full-fidelity output while enabling previews at earlier denoising stages.The section notes that these goals conflict because the final output is the most perceptually consistent preview but arrives last.Source: §4.1 · p. 3

4.1. Previewing With Scene Intrinsics

• Scene intrinsics provide a preview representation in which albedo and depth retain visible object boundaries and scene composition when lighting is discarded.The section describes these channels as lower-frequency signals with larger, colorful patches.Source: §4.1 · p. 3, §3.2 · p. 3
• Linear probes show that scene intrinsics emerge early in diffusion, especially across denoising steps, supporting their use in early previews.The probed targets include base color, depth, surface normals, roughness, metallicity, and RGB.Source: §4.1 · p. 4

4.2. Multi-Branch Preview Predictor

• MSE-based posterior-mean predictions can blur high-motion or spatially complex regions by superimposing plausible trajectories when the noisy-step posterior is multimodal.If unresolved near the end of denoising, these blurred regions may become hallucinated samples.Source: §4.2 · p. 4
• The multi-branch predictor uses independent decoders with branch-wise losses and an ensemble loss to represent multiple modes rather than collapsing them into one prediction.The ensemble averages the branches, and the normal channel uses a directional loss.Source: §4.2 · pp. 4–5
• The multi-branch predictor resolves superposition even at a single NFE, while branch ensembles produce higher-quality previews with clearer edges and diversity than the naive predictor.The mean of the branches also closely matches the ground-truth data mean; the comparison is illustrated on a toy tri-modal dataset.Source: Figure 3 · p. 4, §4.2 · p. 5

5. Generating Variations Using Previews

• Previews at any timestep can support a tree-like workflow, with users moving between denoising levels and exploring variations between branches.The multi-branch decoder produces multi-channel previews in under one second for four-second videos, consistent with the final videos.Source: §5 · p. 5

5.1. Variations Through Stochastic Renoising

• Stochastic renoising adds random noise to a clean latent prediction, preserving generated structure while introducing plausible finer-scale variations.The noise scale follows the original schedule at the timestep immediately after previewing, so users can sample variations without repeating the full diffusion process.Source: §5.1 · p. 5

5.2. Variation Through Latent Steering

• Latent steering uses a trained, frozen preview decoder to optimize later denoising steps toward purposeful variations.The decoder maps intermediate features to intrinsic-map space.Source: §5.2 · p. 5

6.1. Implementation Details

• The evaluation uses Wan 2.1 and a synthetic dataset of 1,000 videos with cached intermediate diffusion features and intrinsic channels.Features are temporally subsampled every fourth frame; input videos are downsampled to roughly 208 × 120 resolution.Source: §6.1 · pp. 5–6

6.2. Baseline Comparison

• At 10% of denoising steps, DiffusionBrowser produces the best PSNR for most channels and is significantly more resource-efficient than the compared methods.Wall-clock time was measured; the table compares models and modalities using PSNR and time.Source: §6.2 · p. 6, Table 1 · p. 7

6.3. Stepwise and Blockwise Preview Evolution

• Scene structure and rough geometry appear by about 10% of denoising steps, while material previews remain stable through generation.Lower blocks capture coarse geometry and color; mid-level blocks capture detailed spatial structure and stable base color and depth, while intrinsic prediction decreases at the last block.Source: §6.3 · p. 6, Figure 6 · p. 6, Figure 7 · p. 7

6.4. Rubber-Like 4D Previsualization

• At 10% of denoising steps, previews support a rubber-like 4D visualization that reveals smooth motion, spatial composition, and overall color palette.The visualization is derived from intrinsic previews computed from noisy intermediate features.Source: §6.4 · p. 6, Figure 8 · p. 7

6.5. Variation Generation

• Stochastic renoising preserves coarse scene structure while allowing finer details to vary; channel-targeted steering varies color or geometry while lighting and texture remain consistent.The authors describe steering as complementary to video editing, rather than a precise editing method.Source: §6.5 · p. 6, Figure 9 · p. 7

6.6. Ablation Study

• The multi-branch decoder has lower MSE and L1 errors across most intrinsic properties than its single-branch counterpart; six layers balance accuracy and computational cost.Shallower and deeper variants show only marginal differences.Source: §6.6 · p. 7

6.7. User Study

• 74.6%, 72.9%, and 76.9% of 35 participants preferred DiffusionBrowser over x0-pred for content predictability, visual fidelity, and scene clarity, respectively.Participants judged the representations alongside a reference video.Source: §6.7 · p. 7

Utilizing Diffusion Features

• Video diffusion features correlate strongly with physical attributes such as depth and albedo, supporting their use to predict multiple intrinsics and probe under-explored representations.The authors connect these correlations to the hypothesis that diffusion implicitly performs inverse rendering.Source: §7 · pp. 7–8

Superposition Problem

• The superposition problem describes blurred intermediate predictions in complex or high-motion regions, attributed to features encoding multiple possible futures.The authors frame it as a superset of hallucination and report that a multi-headed decoder can mitigate it.Source: §7 · p. 8

Limitations and Future Work

• The framework deliberately focuses on scene intrinsics and does not consider text prompts or their interaction with intrinsic previews.The authors identify text-driven conditioning as future work.Source: §7 · p. 8
• Steered intrinsics can dissipate during denoising; the authors attribute this failure to an out-of-distribution issue for their shallow decoder.Figure 12 shows a sphere added at the 20th layer gradually dissolving and deforming in later timesteps.Source: §7 · p. 8, Figure 12 · p. 8

8. Conclusion

• DiffusionBrowser makes coarse-to-fine internal evolution visible, actionable, and efficient, letting users terminate unpromising generations, iterate, and steer without sacrificing final quality.The conclusion characterizes the framework as lightweight and plug-and-play.Source: §8 · p. 8
• The previews expose geometry, layout, and appearance dynamics that govern diffusion behavior, creating opportunities for interpretability and user-driven control.Source: §8 · p. 8
• The authors present DiffusionBrowser as a foundation for future research into controllable diffusion and the structure of generative processes.Source: §8 · p. 8

S1. Analysis: Diffusion Features and Intrinsic Previews

• The decoder produces more stable and coherent geometry than inaccurate pseudo-ground-truth intrinsics, suggesting diffusion features encode intrinsic scene structure reliably.The authors attribute this to the diffusion model’s strong prior and the decoder’s ability to extract structure despite imperfect labels.Source: S1. Analysis: Diffusion Features and… · p. 12, Figure 13 · p. 12
• Nonlinear probes confirm that predictive power saturates early across the network, with the most reliably decodable structure in early or mid-level layers.Deeper decoders improve on linear predictors, but the overall trend remains similar.Source: S1. Analysis: Diffusion Features and… · p. 12, Figure 14 · p. 13
• At 16% of denoising, x0-prediction only then surpasses both feature decoders; the multi-branch advantage over linear predictors widens as noise decreases.At 10% of the timestep, reconstructions already represent scene composition, geometry, and dynamics; the 20% result is more refined.Source: S1. Analysis: Diffusion Features and… · p. 12, Figure 16 · p. 13, Figure 15 · p. 13

S2. Synthetic Training Data Generation

• The synthetic training set uses 1,000 prompts from 40 scene categories, with 25 prompts per category and intrinsics predicted by DiffusionRenderer.Source: S2. Synthetic Training Data Generation · p. 12

S3. Analysis: More on the Toy Problem

• In the toy setting, MSE-trained diffusion predicts posterior means for multimodal clean states at high noise, producing in-between states absent from the data.With low NFE, examples include duplicated or faded dots.Source: S3. Analysis: More on the Toy Problem · p. 12
• The toy multi-branch objective encourages branches toward individual data modes while an ensemble term regularizes their average toward the mean, preserving diversity.Without the ensemble loss, branches collapse toward fewer modes and fail to capture the multimodal distribution; Figure 17 shows separate branch predictions rather than their averaged superposition.Source: S3. Analysis: More on the Toy Problem · pp. 12–13, Figure 17 · p. 14, Figure 18 · p. 14
• 4 dots in expectation for motion-only sequences; under 1-NFE sampling, only the multi-branch decoder consistently reproduces the correct mode structure.For motion and position uncertainty, the average number of boxes is approximately 3.8 because two of nine cases move out of frame at the last frame; the method alleviates the resulting shortfall.Source: S3. Analysis: More on the Toy Problem · p. 13

S4. Details on Latent Steering

• Small gradient-based updates use the learned decoder’s Jacobian to steer intermediate features toward a target intrinsic map.Normals use cosine loss; other modalities use L1 distance, and the stated aim is to propagate preview-level edits into diffusion features.Source: S4. Details on Latent Steering · p. 14
• The proof-of-concept targets shift base color toward another K-means++ cluster, enhance depth edges with Sobel gradients, or invert the normal map’s y-axis.The authors describe these target constructions as intentionally minimal.Source: S4. Details on Latent Steering · p. 14
• Precise color steering is not always possible, and dramatic geometric edits can fail, including depth removal and physically implausible normal flips.The authors attribute failures to possible limits in base-model 3D understanding, decoder capacity, out-of-distribution inputs, and the simple steering method.Source: S4. Details on Latent Steering · p. 14

• Depth and normals reveal coarse scene geometry earlier in denoising than RGB or latents, while base color clarifies layout by omitting lighting.The modalities are produced simultaneously from the same features, enabling cross-reference at any timestep.Source: S5. Benefits of Multi-Modal Previews · pp. 14–15
• Figure 20 compares latent renderings with base color predictions; during early timesteps, intrinsic previews exhibit fewer lighting artifacts and present a cleaner structural representation.Source: S5. Benefits of Multi-Modal Previews · p. 15

• Each trial compared a standard x0-pred preview with an intrinsic-based preview that could include base color, depth, and normals alongside RGB.The study aimed to assess the perceptual usefulness of intrinsic-based previews against the x0-pred baseline.Source: S6. User Study Setup · p. 15
• Participants judged content predictability, visual fidelity, and scene clarity, assessing expected content, artifacts, and scene composition respectively.Source: S6. User Study Setup · p. 15
• 35 participants evaluated 10 examples each, yielding 350 responses per question; they consistently favored intrinsic-based previews across all three criteria.The authors conclude that intrinsic modalities provide more informative and reliable early-stage previews than the x0-pred baseline.Source: S6. User Study Setup · p. 15
• At 4% and 10% of denoising steps, base color previews reveal structural components and color layout more clearly than the latent.Source: Figure 20 · p. 16