Reflection-aware Generative Novel View Synthesis

1 Introduction

• Prior generative NVS methods typically treat mirror images as ordinary views, producing predictions that disagree with mirror content and can bake in mirrors or leak between regions.Finetuning alone does not guarantee models learn physical reflection constraints [25], while large mirror datasets are challenging to collect and large-model finetuning is computationally expensive.Source: §1 · p. 2
• Ref-GeNVS estimates the mirror plane, reflects input camera poses to form virtual views, then uses Mirror-gated attention and Reflection injection to generate target views.Mirror-gated attention conditions on mirror regions of reflected views; Reflection injection adds features from reflected target poses into the mirror region at each denoising step.Source: §1 · p. 2
• On realistic synthetic mirror scenes and a real-scene dataset, Ref-GeNVS produces photorealistic, reflection-consistent views from single and sparse inputs, surpassing recent generative NVS methods.The method uses a multi-view diffusion backbone and requires no finetuning.Source: §1 · p. 2
• Ref-GeNVS is a training-free reflection-aware generative NVS pipeline that treats a mirror image as two complementary views.It is designed to leverage reflected cues during generation without model finetuning.Source: §1 · p. 2

Reflection-aware novel view synthesis

• NeRF- and 3DGS-based NVS often treat reflections as outliers because view-dependent appearances violate assumed multi-view photometric consistency.These methods do not exploit the geometric correspondence induced by mirrors.Source: §2 · p. 3
• Prior reflection methods use diffuse/specular decomposition and environment-lighting estimation [23] [31], or blend real and reflected content from dual radiance fields or reparameterized heads.Source: §2 · p. 3
• Mirror-NeRF, MS-NeRF, TraM-NeRF [44], and MirrorGaussian render mirror-reflected camera rays but focus on reflective objects, not the remaining scene and mirror together.Source: §2 · p. 3
• OrCa and World-from-Eyes use multi-view glossy objects as proxy mirrors for passive non-line-of-sight imaging, whereas Ref-GeNVS assumes an ideal planar mirror and only one or three inputs.Source: §2 · p. 3

Generative novel view synthesis

• Generative NVS methods use diffusion priors or warp-and-inpaint, but are not trained for mirrors; Ref-GeNVS adds mirror information to multi-view diffusion without finetuning [42].Without explicit mirror handling, reflective regions are often misinterpreted as background, yielding views inconsistent with correct reflections.Source: §2 · p. 3

3 Method

• Reflected virtual views are formed by reflecting input camera poses across the estimated mirror plane and horizontally flipping the corresponding images and masks.This supplies complementary reflected evidence while preserving the same-hand coordinate convention as the original inputs.Source: §3 · pp. 4–5

3.1 Preliminary: Multi-view diffusion model

• Ref-GeNVS adds mirror-masked conditioning, Mirror-gated attention, and Reflection injection to multi-view diffusion without finetuning.The backbone uses cross-view attention to exchange information between input and target frames during denoising.Source: §3.1 · p. 5

3.2 Stage 1: Scene generation with Mirror-gated attention

• Stage 1 masks mirror regions and gates reflected-view attention to mirror tokens, preventing non-mirror pixels from influencing target-view synthesis.This addresses mirror misrecognition, misleading evidence from pixels outside mirrors, and inconsistencies between generated scenes and mirror reflections.Source: §3.2 · pp. 5–6

3.3 Stage 2: Mirror surface generation with Reflection injection

• Stage 2 injects reflected-view evidence into the target mirror region and uses SDEdit-style guidance to anchor the surrounding scene and stabilize mirror boundaries.Reflected latents are aligned through decode–flip–encode; guidance anchors the non-mirror region before reflection injection takes over.Source: Figure 3 · p. 7, §3.3 · p. 8

3.4 Automatic preprocess

• The automatic preprocessing pipeline segments mirrors with DAM, predicts per-pixel 3D points, and fits the mirror plane with RANSAC.It is designed to reduce reliance on the manually provided mirror masks assumed by prior mirror-related NVS methods.Source: §3.4 · p. 9

4 Experiments

• Evaluation uses eight synthetic scenes and real Mirror-NeRF data, with input images showing mirrors and test trajectories chosen to reveal mirror-visible content.The synthetic scenes include physically based mirror materials and ground-truth multiview images with camera parameters; comparisons cover sparse-input and single-image NVS settings.Source: §4 · pp. 9–10, Figure 6 · p. 10

4.1 Sparse-input novel view synthesis

• Ref-GeNVS surpasses MVGenMaster [6] and SEVA [61] across DreamSim, CLIP similarity, PSNR, SSIM, and LPIPS in sparse-input NVS.Qualitatively, competing methods misinterpret or ignore reflected regions, while Ref-GeNVS produces more consistent views aligned with the ground truth.Source: §4.1 · pp. 11–12, Table 1 · p. 11, Figure 7 · p. 11

4.2 Novel view synthesis from a single image

• Ref-GeNVS outperforms VistaDream, FlexWorld, and SEVA on DreamSim and CLIP similarity relative to both the input and ground-truth reflected views.With only one input image, it produces more coherent predictions in mirror-revealed areas than methods that struggle to interpret reflected regions.Source: Figure 8 · p. 12, §4.2 · p. 13, Table 2 · p. 13

4.3 Ablation Study

• Adding a reflected virtual view, Mirror-gated attention, and two-step generation progressively improves performance over SEVA in the synthetic sparse-input ablation.The virtual view improves reflection geometry; Mirror-gated attention reduces leakage from non-mirror regions, and two-step generation further aligns the reflected scene with the mirror surface.Source: §4.3 · p. 13, Figure 9 · p. 14

4.4 Applications

• Ref-GeNVS yields more complete, stable, and reflection-consistent 3D reconstructions than FlexWorld [10] and SEVA [61] when generated views are fed into Pi³.Errors in mirror-revealed regions appear as broken geometry in the reconstructions.Source: §4.4 · pp. 13–14, Figure 10 · p. 14
• Ref-GeNVS can use wide-field reflections from distorted mirrors, including convex mirrors, after warping them into a planar-mirror representation without modifying the pipeline.The experiment uses mirror-ball geometry for the warp and applies Ref-GeNVS through Stage 1.Source: §4.4 · pp. 14–15, Figure 11 · p. 15

5 Conclusion

• Ref-GeNVS augments a multi-view diffusion backbone with reflected views by estimating the mirror plane and reflecting camera poses, guiding training-free generation toward reflection-consistent results.Mirror-containing inputs are converted into complementary viewpoints.Source: §5 · p. 15
• Ref-GeNVS generates novel views that better align with mirror-reflected content and expose global scene structure than prior approaches.Source: §5 · p. 15
• Partial mirror observations could help embodied agents infer and predict regions beyond their direct field of view for more informed navigation.The conclusion also notes potential applications in 3D scene generation.Source: §5 · p. 15

Future work

• Future work could extend Ref-GeNVS to a trainable framework with mirror detection and reflected-view conditioning during training to provide explicit physical priors.Naïve training without explicit physical priors is not guaranteed to learn reflection constraints implicitly.Source: §5 · p. 15

• GeonU Kim, Shin Dong-Yeon, and Tae-Hyun Oh are listed as the authors.Source: Supplementary Material · p. 20
• The authors’ listed affiliation is KAIST, with contact addresses geonukim, shindy, and taehyun.oh@kaist.ac.kr.Source: Supplementary Material · p. 20

A Implementation details

• The implementation appendix covers both generation stages, preprocessing automation, and experimental details.Stage 1 also includes boundary inpainting and self-refining sampling as implementation measures.Source: A Implementation details · p. 20, §A.1 · p. 21

A.1 Stage 1: Mirror-gated attention

• Stage 1 generates masked target views from mirror-masked original inputs and reflected inputs.The reflected mirror mask is defined in image space before being aligned to the attention tokens.Source: §A.1 · p. 20
• Mirror-gated attention uses a token-level binary mask aligned to the attention feature map to distinguish tokens in the reflected mirror region.The reflected mirror mask is downsampled to the feature-map resolution, reshaped, and broadcast to the attention tensor layout.Source: §A.1 · p. 20
• Padded query, key, and value tensors encode the mirror mask as a large negative logit bias for non-mirror tokens, using C = 104.The implementation appends a padding channel, computes attention, then removes that channel from the output; it is compatible with PyTorch scaled dot-product attention and FlashAttention without custom kernels.Source: §A.1 · pp. 20–21
• Mirror-gated attention is applied to cross-view 3D and 1D attention and to reflected-view 2D self-attention, restricting contributions to reflected regions.The method also inpaints pixels around mirror boundaries before diffusion and uses self-refining sampling for more stable multi-view generation.Source: §A.1 · p. 21

A.2 Stage 2: Reflection injection

• Stage 2 reuses the Stage 1 multi-view diffusion backbone with Mirror-gated attention, conditioned on original views and reflected inputs.Source: §A.2 · p. 21
• Stage 1 output is not used as a source view; only its non-mirror region guides mirror-pixel synthesis through SDEdit-style guidance.Guidance starts from an intermediate noisy latent of the Stage 1 output.Source: §A.2 · p. 21
• Guidance lasts until t0 = 42 of T = 50 steps, then is removed; a Gaussian-blurred mirror mask blends the injected latent smoothly with surrounding regions.The softened mask is resized to latent resolution.Source: §A.2 · pp. 21–22

A.3 Automation pipeline for preprocess

• Sparse-view mirror-plane estimation is difficult because reflections violate multi-view photometric consistency, while edge-based fitting can fail with occlusion or inaccurate masks.MirrorGaussian uses SfM mirror-edge points with more than 30 input images, a setting unlike Ref-GeNVS’s 1–3 sparse views.Source: §A.3 · p. 22
• Masking mirror regions before per-pixel 3D prediction enables planar mirror estimates across Dust3R [15], Pi3 [16], and DA3, and surface-based estimation is more robust than edge fitting as mask IoU falls.The input mirror masks are estimated with DAM; Fig. S1 illustrates the effect of masking, and Fig. S2 reports robustness under imperfect masks.Source: Figure S1 · p. 22, §A.3 · p. 23, Figure S2 · p. 24
• Without camera poses, the pipeline predicts points and poses from masked inputs, extracts mirror surface points, and fits the mirror plane with RANSAC [4].The described pose-free case uses a feedforward 3D regression method.Source: §A.3 · p. 23
• With given poses, Umeyama alignment reconciles predicted and given camera coordinate systems, rescales predicted depth, and supports RANSAC plane fitting from mirror surface points.Source: §A.3 · p. 23
• For a single image, the pipeline generates a stereo pair with a small parallel camera translation, then uses regression-derived points and relative poses as sparse multi-view inputs to calibrate mirror-plane estimation to the backbone’s scale.The input mirror is masked before generation to avoid artifacts; this addresses the scale mismatch between 3D regression and the generative backbone.Source: §A.3 · p. 23

A.4 Experimental details

• Experiments used one RTX 6000 Ada Generation GPU, Mirror-NeRF and eight synthetic scenes, with three or one input images and nine or fourteen target views, respectively.Mirror-NeRF contains three scenes; preprocessing was automated on synthetic data, while Mirror-NeRF used ground-truth mirror masks and planes. MVGenMaster methods ran at 512 × 512 and SEVA methods at 576 × 576.Source: §A.4 · p. 24

B Robustness on inaccurate mirror detection

• The evaluation progressively pasted random patches onto mirror regions before DAM-based segmentation, then measured mask IoU and novel-view reconstruction performance.Performance was assessed using DreamSim, CLIP similarity, PSNR, and LPIPS after the remaining preprocessing and generation stages.Source: B Robustness on inaccurate mirror… · pp. 24–25
• As mask IoU decreases, both estimation strategies degrade, but surface-based fitting is significantly more robust than edge-based estimation.The surface-based strategy fits a plane using surface points predicted by per-pixel 3D regression; the edge-based strategy estimates it from edge points without mirror masking. Fig. S3 illustrates edge-based plane misestimation and misaligned reflected content under imperfect masks.Source: B Robustness on inaccurate mirror… · p. 25, Figure S3 · p. 25
• At mask IoU around 0.6, surface-based mirror-plane estimation still outperforms SEVA, demonstrating tolerance to imperfect mirror detection.Unlike prior mirror-related NVS methods such as Mirror-NeRF and MirrorGaussian, which assume given mirror masks, this pipeline aims to relax that assumption.Source: B Robustness on inaccurate mirror… · p. 25

C Limitations

• Very small, partially visible, or heavily occluded mirrors can leave geometric cues too scarce or unstable for 3D point regression, producing inaccurate mirror planes and misaligned generated views.The failure case in Fig. S4 shows misalignment with ground truth at the same camera poses.Source: C Limitations · pp. 25–26, Figure S4 · p. 26
• Mask-tolerant estimation remains relatively stable as mirror-mask IoU decreases, but preprocessing depends on Depth Anything 3 cues and may benefit from more advanced 3D regression methods.When reliable geometric cues are insufficient, mask robustness does not prevent mirror-plane estimation failure.Source: C Limitations · p. 26

Single image novel view synthesis

• On real-world single-image NVS, Ref-GeNVS leverages reflected views to generate mirror-revealed content, whereas SEVA struggles to exploit reflected evidence.The results use 360-degree orbit target-camera poses; these datasets provide no ground-truth images of the scene behind the mirror.Source: D Additional experimental results · p. 26, Figure S6 · p. 27

Sparse images novel view synthesis

• Ref-GeNVS generates reflection-consistent scene content in sparse-image NVS, unlike prior methods, on real and synthetic scenes.The qualitative comparisons include the real dataset and the synthetic dataset.Source: D Additional experimental results · p. 26, Figure S5 · p. 27

Region-wise evaluation

• Region-wise PSNR shows a larger performance gap in reflected regions than across the full scene, while mirror-surface generation also improves.The evaluation masks reflected regions and mirror surfaces; reflected-region PSNR can remain low because of large camera-pose gaps between target and reflected input views.Source: D Additional experimental results · p. 28, Table S1 · p. 28

Inference time

• Ref-GeNVS based on SEVA takes approximately 397 seconds per scene, compared with about 130 seconds for SEVA, on an NVIDIA RTX 6000 Ada Generation GPU.The added time comes from automated preprocessing and the two-step generation pipeline, a computational trade-off for reflection-consistent synthesis without additional training.Source: D Additional experimental results · p. 28