Source-linked AI summary
Unravel: Recovering an Interactive 3D Scene from a Single Image
Hui Shan, Keyang Luo, Binjia Zhou, Ming Li, Kai Li, Xin Wang, Zhenyang Li, Zesong Yang, Weikai Chen, Runze Zhang, Xin Wang, Yanwei Fu, Zhen Chen, Xiangru Huang
TL;DR
Single-image reconstruction must recover movable objects and physically meaningful scenes, but existing approaches leave objects entangled or depend on costly learned priors. Unravel iteratively removes and verifies exposed objects, then recomposes them using point-cloud anchors and physical settling. Experiments report faithful interactive scene reconstruction, with improved asset fidelity and more faithful layouts in the reported comparisons.
Problem
Single-image methods may leave geometry without explicit object decomposition, while object-centric generators rely on costly learned priors and can struggle with occlusion-driven shape errors.
Source: §1 · p. 2Method
Unravel iteratively removes and verifies exposed objects, extracts independent assets from best-observed snapshots, and recomposes them using point-cloud provenance, one-sided extent constraints, and physical settling.
Source: §1 · pp. 2–3Results
Experiments across diverse indoor scenes demonstrate faithful interactive scene reconstruction, with improved asset fidelity over direct reconstruction and more faithful layouts than feed-forward and agentic baselines in qualitative comparisons.
Source: §1 · p. 3Takeaways & Limitations
The recovered scene is intended for rearrangement: its objects can be moved or replaced while the scene is re-settled into a physically valid state.
Source: §1 · p. 2, §3.1 · p. 5In highly cluttered scenes, current image-editing models may alter remaining objects, and accumulated drift can disrupt point-cloud alignment and asset placement.
Source: §5 · p. 10
Abstract
from arXiv · showhide
We present Unravel, a training-free framework that turns a single image into an interactive 3D scene, where every object is an independent and movable asset. Existing generators either produce one entangled geometry in which no object can be moved, or predict all objects in one feed-forward pass, where a wrong guess cannot be detected or repaired. Unravel instead takes a different approach: it first takes the scene apart, and then puts it back together. Starting from the input image, an agentic harness that iteratively peels the image layer by layer, and adopts point cloud as major modality for spatial relationship recovery. Specifically, a vision-language model plans which objects are fully visible, an image editor removes them and restores the occluded regions. Every edit is verified before being incorporated into the reconstruction. This process gradually turns occluded parts of the scene into direct observation, allowing objects to be reconstructed when they are best exposed rather than hallucinated from the original image alone. The scene is then rebuilt in reverse removal order, so that the supporting objects are naturally placed before the objects they support. Since a single-view image only reveals part of an object, we treat the observed points as a lower bound on its size: an asset may grow to cover its points, but can not shrink away from them. Finally, we physically settle the reconstructed scene to remove floating objects and interpenetrations, followed by an agentic audit that corrects remaining placement errors. Extensive experiments on diverse indoor scenes show that Unravel recovers more objects, places them more faithfully, and produces scenes that are more physically-plausible than existing methods.
1 INTRODUCTION
Single-image scene reconstruction must recover independently movable objects while preserving physical meaning under rearrangement. Unravel separates object recovery from recomposition, using verified deconstruction and geometric anchors to build and refine the scene.
1 INTRODUCTION
- Existing methods either omit explicit object decomposition or rely on costly learned priors, while occlusion-driven shape errors can distort assets and propagate into layout.Because object recovery and placement are coupled, these errors are difficult to isolate and correct.Source: §1 · p. 2
- Unravel uses a training-free, verified layer-wise process to remove exposed objects, restore freed image regions, and extract independent assets from their best-observed snapshots.A verifier checks each edit before it contributes to reconstruction.Source: §1 · pp. 2–3
- The deconstruction record anchors recomposition: per-round point maps retain round-and-pixel provenance, and reverse removal order guides assembly from supporting objects to those they carry.Instance masks associate observations with asset clusters for placement.Source: §1 · pp. 2–3
- One-sided extent constraints treat observed spans as lower bounds; physical settling and a final audit refine placement while reducing floating and interpenetration.The constraints avoid forcing a complete asset to match a partial surface.Source: §1 · pp. 2–3
- Extensive experiments across diverse indoor scenes demonstrate that Unravel faithfully reconstructs interactive 3D scenes from a single image.Ablations show improved asset fidelity over direct reconstruction, while qualitative comparisons with feed-forward and agentic baselines show more faithful layouts.Source: §1 · p. 3
2 RELATED WORK
Related work spans feed-forward geometry recovery, object-asset generation, and interactive scene synthesis. Unravel builds on these lines while linking iterative deconstruction to physically constrained recomposition.
2.1 FEED-FORWARD 3D RECONSTRUCTION
- Feed-forward methods recover per-pixel geometry but lack explicit object decomposition, leaving occluded regions incomplete and objects inseparable.Examples include DUSt3R (Wang et al., 2024), VGGT (Wang et al., 2025a), and MoGe (Wang et al., 2025c; Wang et al., 2026; Kong et al., 2026).Source: §2.1 · p. 3
2.2 3D OBJECT ASSET GENERATION
- Image-to-3D and part-aware models provide strong priors for individual assets, but extending them to complete scene generation remains challenging.Examples include TRELLIS.2 (Xiang et al., 2026), PartCrafter (Lin et al., 2026), and HoloPart (Yang et al., 2026).Source: §2.2 · p. 3
2.3 INTERACTIVE 3D SCENE GENERATION
- Interactive scene methods include text-driven layout, agentic refinement, and single-image synthesis, but semantic instructions underspecify reference-image geometry and some methods do not target independent editing.Unravel links verified deconstruction to physically constrained recomposition, using provenance anchors and reverse removal order for support-aware assembly.Source: §2.3 · pp. 3–4, Figure 3 · p. 4
3 METHODS
Unravel recovers an interactive scene by deconstructing a single image into object assets, then recomposing them using fused point-cloud evidence. Its method combines layer-wise removal and reconstruction with support-aware placement, one-sided size constraints, and physical settling.
3 METHODS
- Unravel deconstructs a scene before recomposing it, extracting independent assets from best-observed snapshots and using aligned, fused point clouds to retain object locations.Recomposition places assets in reverse removal order, registers them to point clusters, and physically settles the assembly.Source: §3 · p. 4
3.1 PROBLEM FORMULATION
- The recovered scene comprises a floor-and-wall shell and independently placed assets, each with rotation, translation, and axis-wise scale, plus recorded support relations for later rearrangement and settling.The scale is resolved axis by axis because a single view bounds each axis's extent only from below.Source: §3.1 · pp. 4–5
3.2 AGENTIC LAYER-WISE DECONSTRUCTION
- A VLM planner selects fully visible objects for removal, an editor inpaints freed regions, and a verifier checks edits before they contribute to reconstruction.Each asset is extracted immediately before removal from its best-observed snapshot; fixed-camera reconstructions are aligned and fused, with masks and validity filters controlling point contributions.Source: §3.2 · pp. 5–6
- Layer-wise point attribution resolves occlusion ambiguity by labeling each point according to its round, pixel, instance masks, and removal order.Points assigned to an asset form its cluster, while room-labeled points are used to fit the environment shell.Source: §3.2 · p. 6
3.3 SUPPORT-AWARE SCENE RECOMPOSITION
- Recomposition assembles assets in descending removal order so supports are placed before the objects resting on them, then registers each asset to its point cluster under one-sided extent constraints.Under-observed axes must extend at least as far as observed points, so registration may grow an asset but cannot shrink it away from those points; physical settling and a final audit address placement errors.Source: §3.3 · pp. 6–7
4 EXPERIMENTS
The experiments compare Unravel with feed-forward and agentic baselines, then assess how iterative deconstruction affects masks, assets, and scene geometry. Qualitative comparisons show Unravel preserving layouts closely, while the ablation measures additional observations obtained through deconstruction.
4.1 EXPERIMENTAL SETUP
- The evaluation uses off-the-shelf components and compares feed-forward predictors with Astra’s direct-generation and shared-asset placement settings.The shared-asset setting gives Astra Unravel’s meshes and masks, so it estimates asset scale and pose rather than generating meshes.Source: §4.1 · pp. 7–8
4.3 ABLATION STUDY
- Direct-reconstruction mask areas average 77.2% of Unravel’s for objects removed after round one, including 85.0% for round two and 60.4% for round four.Objects removed after the first round account for roughly one third of the test set.Source: §4.3 · p. 8, Table 1 · p. 8
4.2 QUALITATIVE COMPARISONS
- Across the compared scenes, Unravel preserves the input layout more closely than the feed-forward and agentic baselines.The shared-asset Astra comparison isolates placement from asset quality, and its red-boxed discrepancies remain despite using the same meshes.Source: §4.2 · p. 8, Figure 5 · p. 9
4.3 ABLATION STUDY
- Fusing four to eight rounds raises mean points per scene from 1.53 million to 2.81 million; direct reconstruction covers 76.9% of fused clusters on average.For evaluated objects, coverage is below 70% for 10.1% and below 50% for 3.8%, with a minimum of 11.9%.Source: Table 1 · p. 8, §4.3 · p. 9
5 LIMITATIONS AND FUTURE WORK
Unravel’s performance is constrained by current image-editing models, especially in highly cluttered scenes where edits can drift across rounds and affect reconstruction.
5 LIMITATIONS AND FUTURE WORK
- In highly cluttered scenes, image edits may alter remaining objects’ appearance or position, and drift can accumulate across rounds.The accumulated drift can disrupt point-cloud alignment and asset placement.Source: §5 · p. 10
- Verification improves editing reliability, but minor errors may persist.Source: §5 · p. 10
- The authors suggest stronger editors with geometric feedback and uncertainty-aware checks that trigger targeted retries.Source: §5 · p. 10
A.1 ACCURATE AND STABLE BOUNDING BOX LOCALIZATION
The localization procedure pairs global target boxing with per-target checks and redetection, then protects movable objects that remain unboxed.
A.1 ACCURATE AND STABLE BOUNDING BOX LOCALIZATION
- The vision-language model boxes all named targets in one call, using tight boxes that touch each object’s outermost pixels.Boxes with IoU at least τdup are treated as duplicate boxes for one object.Source: §A.1 · p. 13
- An independent local probe confirms a global box when their IoU reaches τp; otherwise, local detection replaces the drifted box with the median of agreeing samples.If probing or local detection returns no box, the global box is kept; experiments use n = 3, τdup = 0.85, τc = 0.7, and τp = 0.5.Source: §A.1 · p. 13
- After verification, the annotated plan image is checked again, and any movable object still without a box is added as an object to protect.A missed object is therefore protected rather than deleted by mistake.Source: §A.1 · p. 13
A.2 REGISTRATION AND SETTLING DETAILS
Registration treats observed point clusters as lower bounds on object size, then settling enforces support and non-overlap while limiting movement from observed positions.
A.2 REGISTRATION AND SETTLING DETAILS
- An under-observed axis may only grow, while a well-observed axis may contract; corrections are kept only when they do not worsen the point-cluster fit.Uniform scaling is used when all three axes indicate the asset is too small, preserving its proportions.Source: §A.2 · p. 13
- An object is supported when its bottom lies within a small tolerance of the highest floor or object-top surface beneath it.The floor is always a candidate support surface, and the experiments use ϵs = 3 cm.Source: §A.2 · p. 13
- The movement budget keeps settling corrections near the observed cluster: a misplaced asset may move toward its observation but never farther away.The experiments set ϵ to 0.3 m horizontally and 0.15 m vertically.Source: §A.2 · pp. 13–14
A.3 ASSET SCORING PROTOCOL
The asset-scoring protocol compares Unravel meshes with direct reconstructions against the same best-observed image references. Human raters and GPT-5.5 score fidelity independently using a shared rubric.
A.3 ASSET SCORING PROTOCOL
- The evaluation covers 160 objects from five test scenes, with 15 human participants rating each object’s direct-reconstruction and Unravel meshes.Each mesh pair is rendered from the same four viewpoints at 90-degree intervals and shown side by side in random order.Source: §A.3 · p. 14
- Raters compare both meshes against a crop of the object from its best-observed round, without knowing which pipeline produced either mesh.The crop is the same reference for both meshes in each pair.Source: §A.3 · p. 14
- The 1–10 rubric ranges from wrong or incomplete shapes to meshes faithful in shape and structure, including parts occluded in the input image.GPT-5.5 and human raters score independently; Table 1 reports their mean scores over all 160 objects, and Fig. 6 shows representative examples.Source: §A.3 · p. 14
A.4 ADDITIONAL SCENE RECONSTRUCTION RESULTS
The reconstructions preserve the main furniture arrangement in a cluttered comparison and across additional indoor scenes. Further examples include six real photographs whose renderings retain their main furniture arrangements.
A.4 ADDITIONAL SCENE RECONSTRUCTION RESULTS
- In a cluttered-image comparison, Unravel more closely preserves the main furniture arrangement, while Seeing Through Clutter produces fragmented geometry and misplaced objects.The comparison uses Seeing Through Clutter’s open-source configuration with its planner upgraded to GPT-5.5 to match Unravel.Source: §A.4 · p. 14
- Across sixteen additional scenes, the same pipeline and settings yield plausible image edits and reconstructed scenes that preserve the input layout.The examples span kitchens, living rooms, offices, and specialty spaces; Figures 8–10 show deconstruction traces, extracted assets, and multi-view renderings.Source: §A.4 · p. 15, Figure 8 · p. 16, Figure 9 · p. 17
- Six real indoor photographs show reconstructions whose renderings retain the main furniture arrangements.The photographs depict hotel interiors, a residential living room, and a clinic waiting room.Source: §A.4 · p. 15, Figure 11 · p. 19