Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections

1. Introduction

• Diffusion text-to-image and inpainting methods struggle to produce plausible, consistent, and controlled mirror reflections.The authors identify realistic, controllable mirror-reflection generation as an unresolved challenge.Source: §1 · p. 2
• The paper formulates reflection generation as image inpainting, using input-image cues while allowing greater control over mirror placement.This formulation is intended to help reflections take cues from the input image.Source: §1 · p. 2
• SynMirror contains 198,204 rendered samples from 66,068 unique 3D objects, while MirrorBench is a subset for benchmarking reflection generation.The dataset addresses existing datasets’ generic mirror backgrounds and insufficient scale for generative-model training.Source: §1 · p. 2
• MirrorFusion significantly outperforms state-of-the-art diffusion-based inpainting methods on SynMirror.The paper reports extensive quantitative and qualitative evaluations.Source: §1 · p. 2
• MirrorFusion is a depth-conditioned inpainting method proposed to address geometrically inconsistent reflections produced by fine-tuned inpainting models.The authors hypothesize that additional cues such as depth maps could help alleviate the geometry problem.Source: §1 · p. 2

2. Related Work

• Text-to-image diffusion models generate photorealistic images from prompts but do not produce realistic mirror reflections.Diffusion models have also been extended to video, audio, and text.Source: §2 · p. 2
• Diffusion inpainting methods modify denoising or model masked-image features, yet cited approaches do not perform well for mirror reflections.The passage notes that standard masked-region sampling does not generalize to complex scenes and shapes.Source: §2 · p. 3
• Reflection research covers image removal, reflective-surface detection, and 3D mirror-reflection reconstruction for novel-view synthesis, rather than controlled object reflections in input images.The paper identifies its target as generating controlled, photorealistic, geometrically consistent reflections for an object in an input image.Source: §2 · p. 3

3. SynMirror: A synthetic dataset of mirror reflections

• SynMirror includes diverse mirror types, objects, camera poses, HDRI backgrounds, and floor textures, and is more than six times larger than existing mirror datasets combined.The paper identifies limited scale and object diversity in existing datasets as obstacles to incorporating mirror-reflection priors into diffusion models.Source: §3 · p. 3, Table 1 · p. 3
• MirrorBench is a challenging subset for evaluating mirror-reflection generation, while SynMirror can also support monocular depth estimation and novel-view synthesis benchmarks.SynMirror includes color images, instance segmentation masks, depth maps, and normal maps.Source: §3 · p. 3

3.1. Dataset Generation and Processing

• SynMirror combines 58,115 filtered Objaverse assets with all 7,953 ABO objects, using objects, varied mirrors, and floor textures in composed scenes.Objects are normalized to fit a unit cube and rotated around the y-axis to increase pose diversity.Source: §3.1 · p. 3
• The rendering pipeline samples varied cameras and HDRI environments, then produces RGB images plus depth, surface-normal, and semantic annotations.Scenes include sampled 3D objects, mirror planes, and diverse floor textures; renderings use 512 × 512 resolution and 1024 Cycles samples.Source: Figure 3 · p. 4, §3.1 · p. 4

4.1. Preliminaries

• Diffusion models iteratively denoise noisy samples, while latent diffusion performs this process in a compressed space for efficient training and faster high-resolution inference.The denoiser is trained with a mean-square loss conditioned on timestep and optional inputs.Source: §4.1 · p. 4

4.2. MirrorFusion

• MirrorFusion treats reflection generation as inpainting and conditions it on depth maps to provide geometric cues for reflections consistent with scene and object structure.Given an input scene and mirror mask, it fills the masked region with the reflection.Source: §4.2 · p. 4

4.2.1 Model architecture

• A cloned conditioning branch receives image, mask, and depth latents; zero-convolution layers inject its features into a frozen generation U-Net, whose decoder produces the reflection.Only the conditioning model is updated during training.Source: Figure 4 · p. 5, §4.2.1 · p. 5, §4.2 · p. 5

4.2.2 Depth conditioning

• Noisy, masked-image, mask, and depth inputs condition the model, while corresponding conditioning features guide each generation U-Net layer through zero-convolutions.The generation U-Net remains an unaltered text-to-image diffusion model that predicts a cleaner latent.Source: §4.2.2 · p. 5
• With depth conditioning, MirrorFusion preserves object structure in examples where SynMirror-finetuned BrushNet produces a distorted baseball and asymmetric chair.Figure 5 marks the distorted shapes in red and preserved structure in green.Source: §4.2.2 · p. 5, Figure 5 · p. 6

4.2.3 Depth Normalization

• Depth normalization targets relative mirror-to-scene distance, clipping input depth at the mirror-mask maximum plus Δdepth=0.5; inference uses a predefined reflection mask and Marigold-estimated scene depth.The authors also report compatibility with alternative monocular depth estimators.Source: §4.2.3 · pp. 5–6

5. Experiments & Results

• MirrorBench evaluates 2,991 images across 1,497 known and 1,494 unknown samples, using preservation, reflection-quality, geometry, and text-alignment metrics.The baselines include zero-shot Stable Diffusion Inpainting, PowerPaint, and BrushNet, plus BrushNet fine-tuned on SynMirror.Source: §5 · p. 6

5.1. Qualitative Results

• MirrorFusion preserves chair structure and reflection shape, and places objects accurately in the mirror where BrushNet-FT shows structural or positional errors.The examples include a lawn chair, swivel chair, and additional mirror-placement comparisons.Source: Figure 6 · p. 6, §5.1 · p. 8
• Zero-shot baselines fail or place reflections incorrectly; BrushNet-FT improves plausibility but has geometric and size errors, while MirrorFusion produces realistic reflections.Examples include a misplaced lipstick, duplicate reflections, and incorrect reflected-object sizes.Source: §5.1 · pp. 6–7, Figure 7 · p. 7
• On held-out GSO objects, MirrorFusion produces more photorealistic and geometrically accurate reflections than BrushNet-FT, whose examples include a floating bowl and oversized reflected bag.Source: §5.1 · p. 8, Figure 9 · p. 8
• Across a continuous viewpoint trajectory, the sofa-seat reflection remains consistent and the floor reflections retain high fidelity.Source: §5.1 · p. 8, Figure 8 · p. 8

5.2. Ablation studies

• On MirrorBench, MirrorFusion with depth cues outperforms BrushNet-FT across image-generation and reflection-quality metrics.The reported values average representative outputs selected by mask SSIM from four random-seed generations per test sample.Source: Table 2 · p. 8, Table 3 · p. 8

• SynMirror is a large, diverse dataset for training generative models to produce realistic mirror reflections.The paper also presents MirrorBench for benchmarking reflection generation.Source: §6 · p. 8
• MirrorFusion conditions an inpainting method on depth maps to generate geometrically accurate mirror reflections.Source: §6 · p. 8
• Extensive qualitative and quantitative results on MirrorBench show MirrorFusion's superior performance compared with other methods.Source: §6 · p. 8

A. Dataset

• SynMirror contains 198,204 rendered images from 66,068 objects sourced from Objaverse and ABO, with captions from Cap3D.The scenes vary in 3D objects, floor textures, and HDRI backgrounds.Source: A. Dataset · p. 12

A.1. Filtering out Spurious objects

• The filtering procedure removes objects whose material-node configuration indicates their reflections will not appear in the mirror.The described check looks for a Mix-Shader node with a Fac input linked to a Light Path node; the filtered list is to be released with the dataset.Source: §A.1 · p. 12

A.2. Preparation of MirrorBench

• MirrorBench samples around 1,000 SynMirror objects, with three rendered samples per object, for 2,991 total samples.Source: §A.2 · p. 12
• MirrorBench separates 1,494 samples from categories absent during training as “Unknown” from 1,497 “Known” category samples.The known samples include renderings from around 250 Objaverse and around 250 ABO objects.Source: §A.2 · p. 12

B.1. Training Details: MirrorFusion

• MirrorFusion follows BrushNet’s architecture, freezes the generation U-Net, and updates the conditioning network while training on SynMirror.Training used 512 × 512 images, AdamW at learning rate 1e-5, 20,000 steps, and 8 NVIDIA A6000 GPUs; the 15,000-step checkpoint was selected.Source: §B.1 · p. 15
• Making MirrorFusion’s generation U-Net trainable improves qualitative results, but nearly doubles VRAM requirements and training time.This variant is called MirrorFusion*; the reported comparison uses its 17,000-step checkpoint.Source: §B.1 · p. 15

B.2. Training Details for Baseline Methods

• BrushNet-FT is trained with the input mask and masked image using MirrorFusion’s hyperparameters, without random text-prompt dropping.The authors report that Stable Diffusion v1.5 initialization outperformed pretrained BrushNet initialization.Source: §B.2 · p. 15

B.3. Inference Details

• Inference uses classifier-free guidance scale 7.5 and the UniPC scheduler for 50 timesteps across experiments.Source: §B.3 · p. 15
• On MSD images and smartphone-captured scenes, BrushNet-FT fails to generate accurate reflections, while MirrorFusion generates plausible mirror reflections.Source: Figure 13 · p. 16

C.1. More Results on Google Scanned Objects (GSO)

• On Google Scanned Objects, MirrorFusion* consistently generates accurate object and floor reflections, while BrushNet-FT shows floor, appearance, and structure errors.The appendix also reports that MirrorFusion is significantly better than Adobe Firefly in a qualitative comparison.Source: §C.1 · p. 16, Figure 14 · p. 16

C.2. Results on real-world scenes.

• On MSD and hand-held smartphone images, MirrorFusion produces better reflections than baselines; smartphone examples use manually annotated mirror masks and Marigold-inferred depth.The MSD examples use dataset-provided ground-truth mirror masks.Source: §C.2 · p. 16

C.3. Comparison with Commercial Products.

• MirrorFusion significantly outperforms commercial solutions such as Adobe Firefly in the presented comparisons.The examples compare reflections of a liquor bottle and a handled kettle-ball.Source: §C.3 · p. 17

C.4. Robustness to pre-trained monocular depth estimation methods

• Reflection generation varies minimally between Marigold and DepthAnythingV2, indicating robustness to the selected pre-trained monocular depth estimator.Marigold is used in all experiments.Source: §C.4 · p. 17, Figure 16 · p. 18

C.5. More Qualitative Comparisons

• In additional comparisons against zero-shot and SynMirror-fine-tuned baselines, MirrorFusion produces better reflections while preserving object appearance fidelity.The comparisons are described as consistent with findings in the main paper.Source: §C.5 · p. 17

Limitations

• Synthetic-data training leaves room to improve reflections of highly complex objects and scenarios, as well as results on real-world images.The authors suggest more advanced photorealistic simulators or large-scale real-world image collections as possible future directions.Source: D. Limitations and Social Impact · p. 18

Social Impact

• Because the method uses diffusion-based generative models, it could be exploited to spread misinformation.The authors call for responsible use.Source: D. Limitations and Social Impact · p. 18

E.1. Results from recent T2I methods

• Standalone text-to-image methods are inadequate for generating controlled, realistic mirror reflections.The appendix illustrates this with results from Stable Diffusion 3.Source: §E.1 · p. 18, Figure 19 · p. 22

E.2. Text prompts used in the experiments

• The appendix lists object-specific text prompts used to generate results for several main-paper figures.The listed prompts include descriptions for Figures 1, 5, and 6.Source: §E.2 · pp. 18–19

E.3. Generation of Segmentation Masks for computing metrics

• Reflection-mask IoU is computed using SAM masks initialized with seed points and a rough bounding box; camera-pose-based seed mapping addresses viewpoint variation.Ground-truth masks are cached to accelerate evaluation.Source: §E.3 · p. 19
• BrushNet-ZS fails to produce plausible reflections, while BrushNet-FT can produce plausible results but retains shape or pose errors; MirrorFusion* mitigates some illustrated issues.The examples include missing object parts and mismatched structure or pose.Source: Figure 17 · p. 20, Figure 18 · p. 21