Source-linked AI summary
Finetuning Prompt-Based Image Editor for Geometry-Consistent Room Defurnishing
Prakhar Kulshreshtha, Brian Pugh, Zachary Nussdorfer, Salma Jiddi
TL;DR
Room defurnishing for XR requires removing furniture while preserving realistic appearance and room geometry, but prompt-based editors lack explicit geometric constraints. The paper finetunes a FLUX.2 LoRA on furnished-empty room pairs; it achieves the best reported texture fidelity and geometric consistency among the compared methods, though its image-only design does not guarantee multiview or temporal consistency.
Problem
Defurnishing must inpaint removed furniture while preserving room geometry, yet prompt-based editors lack explicit geometric constraints.
Source: §1 · p. 1Method
The paper finetunes a FLUX.2 LoRA on 2,500 furnished-empty room pairs for single-image, prompt-based defurnishing.
Source: §1 · p. 1, §2.3 · p. 2Results
Takeaways & Limitations
The single-photo inpainter can serve as a generative inpainting module in future depth-aware and 3D XR pipelines.
Source: §1 · p. 1The image-only method uses no explicit 3D conditioning or persistent scene representation, so multiview and temporal consistency are not guaranteed; inference takes up to 5 seconds per image.
Source: §3 · p. 2
Abstract
from arXiv · showhide
Erasing all furniture from a single room image to generate its empty counterpart is valuable for XR applications such as diminished reality and virtual staging. The challenge is twofold: inpainting the furniture regions with realistic content, and doing so without distorting the room's 3D geometry. Prompt-driven image editors, such as FLUX.2, handle realism impressively, but their structural faithfulness is inferior to explicit layout conditioned inpainting. To improve geometric consistency, we propose a training recipe for finetuning a LoRA on 2,500 furnished-empty room pairs. FLUX.2 with our LoRA adaptor significantly improves geometric consistency while retaining the high texture fidelity. This single-photo inpainter can be an effective building block for depth-aware and 3D-consistent XR pipelines.
1 INTRODUCTION
Room defurnishing must remove furniture and shadows while preserving the room geometry needed for XR. The paper evaluates prompt-based editing and adapts FLUX.2 with LoRA for more geometrically consistent single-image defurnishing.
1 INTRODUCTION
- Defurnishing removes furniture from a room representation while requiring realistic inpainting and faithful walls and floors, especially for AR/MR objects anchored to the environment.The paper focuses on prompt-based editing from a single RGB image and positions its LoRA adaptation as a possible inpainting module for future depth-aware and 3D XR pipelines.Source: §1 · p. 1
- The central question is whether prompt-based editors preserve room geometry during defurnishing and how their geometric consistency can be improved.Layout-guided inpainting is not robust to shadows and can introduce artifacts in non-planar regions, while prompt-based editors lack explicit geometric constraints.Source: §1 · p. 1
- FLUX.2 with the LoRA adaptor achieves the best texture fidelity and geometric consistency among the three compared methods.The adaptation uses 2,500 furnished-empty room pairs.Source: §1 · p. 1
2 METHOD
The method benchmarks three furniture-inpainting approaches using a masked RGB room image, then evaluates texture fidelity and geometric consistency. Its FLUX.2 LoRA is trained on furnished-empty pairs, while the baselines use layout conditioning or prompt-only editing.
2 METHOD
- Three approaches take an RGB image and binary furniture mask as input: layout-guided Stable Diffusion, FLUX.2, and FLUX.2 with LoRA.Masks may be user-defined or derived from segmentation models; the LoRA dataset contains 2,500 in-house furnished-empty room pairs.Source: §2 · p. 1, Figure 3 · p. 2
2.1 SD + LoRA + ControlNet
- Stable Diffusion 1.5 inpainting receives rendered depth and normal maps through two ControlNets, each weighted α = 0.5.A 3D layout extractor estimates wall and floor planes; a LoRA is also finetuned because the base model often leaves furniture shadows.Source: §2.1 · p. 2
2.2 FLUX.2
- Prompt-only FLUX.2 removes masked furniture and shadows while preserving unmasked regions, serving as the no-geometric-conditioning reference.The approach applies a magenta mask to the image and uses the distilled four-step model.Source: §2.2 · p. 2
2.3 FLUX.2 + LoRA
- FLUX.2’s LoRA is finetuned on 2,500 furnished-empty pairs and applied at inference to the distilled four-step model.Training uses rank 64, alpha 32, and 3,000 steps; differential guidance focuses on inpainted areas.Source: §2.3 · p. 2
2.4 Evaluation metrics
- Texture fidelity is measured by LPIPS on 300 test pairs, while geometric consistency uses Layout Error on 100 furnished images with annotated layout boundaries.Layout Error measures annotated boundary pixels’ distance to the nearest predicted edge, with threshold τ capping the penalty for severely missed boundaries.Source: §2.4 · p. 2
3 RESULTS AND CONCLUSION
The LoRA adaptation gives FLUX.2 the strongest reported combination of texture fidelity and geometric consistency among the evaluated methods. The image-only approach does not guarantee multiview or temporal consistency and is currently intended for offline preparation.
3 RESULTS AND CONCLUSION
- FLUX.2 with the LoRA adaptor achieves the best performance on both texture fidelity and geometric consistency.The adaptation specializes the generalist editor for defurnishing by learning geometric cues such as vanishing lines, room-element sizes, and scene layout.Source: §3 · p. 2
- Layout-guided SD+LoRA+ControlNet has lower Layout Error than FLUX.2 but worse LPIPS, while unadapted FLUX.2 can distort room geometry.The passage attributes the geometry distortion to FLUX.2 being a generalist model not optimized to preserve geometry.Source: §3 · p. 2
- Because the method uses only one image and no persistent scene representation, multiview and temporal consistency are not guaranteed.Inference takes up to 5 seconds per image, so the current use is offline scene preparation rather than real-time XR.Source: §3 · p. 2