PaperlayerView exact revision on arXiv

Source-linked AI summary

ProClosure: Hierarchical Room-Object Assignment using Progressive Boundary Closure from Monocular Video

Vinoth Kumar Muthuraj, Soumyadeep Banik, Kushal Sharma, Hardik Jain

arXiv:2609.12614v1cs.RO

TL;DR

Incomplete monocular-video boundaries make doorways and unobserved wall gaps geometrically indistinguishable for room recovery. ProClosure closes regions progressively and assigns objects using their reconstructed extent; over ten floors it improves object-to-room ARI against HOV-SG, though wide openings can merge rooms.

  • Problem

    Monocular video records walls only where observed, leaving doorway gaps and unobserved wall gaps geometrically indistinguishable for room partitioning.

    Source: §1 · p. 2

  • Method

    ProClosure progressively thickens mapped boundaries and freezes each enclosed room, then uses reconstructed object extent and visibility to assign objects.

    Source: §1 · p. 2, §6 · p. 12

  • Results

    0.696 against 0.488 ARI: ProClosure improves object-to-room assignment over HOV-SG across ten floors, with a significant paired-test difference.

    Source: §6 · p. 13

  • Takeaways & Limitations

    Boundary gaps can be treated by width rather than classified as doorways, while visibility helps keep rooms distinguishable for object assignment.

    Source: §1 · p. 2

    On an open-plan floor, wide openings joined spaces before they separated, and the ten-floor paired sample supports only limited claims.

    Source: §5.1 · p. 10, §5.4 · p. 12

Abstract

from arXiv · show

A 3D scene graph groups objects into rooms. When a robot is asked to fetch an object from the kitchen, that grouping is what tells it where to look. An object recorded in the wrong room is not retrievable by a query naming the correct room. We introduce Progressive Boundary Closure, which recovers room layer from a monocular RGB video. A SLAM front end and an open-vocabulary segmenter supply a structural point cloud, camera trajectory and object tracks. The cloud is rasterised into a top-down map, rooms are recovered from it, and each object takes the room holding most of its extent. The difficulty lies in the map itself. Walls are recorded only where the camera looked, so a gap in the boundary may be a doorway or a stretch of wall that was never observed; nothing distinguishes the two. Prior methods treat both as passages, merging rooms that should remain separate. We observe that both require the same treatment: a room should not extend across either, so both are closed and need not be distinguished. Such an opening closes under a small amount of boundary growth, and few sightlines cross it, so points in different rooms rarely see one another. We use the first to recover rooms and the second to assign objects to them. Rooms are obtained by Progressively thickening the boundary inward and freezing each free-space region once it becomes enclosed, so every opening seals at its own scale rather than at a radius fixed in advance. Camera poses are used as seeds, which removes the sampling heuristic and makes the segmentation deterministic. Over 10 floors of 6 HM3D-Semantics scenes, scored against HOV-SG on identical top-down maps, we recover 74 rooms for 72 annotated regions (HOV-SG: 44), raising room F_1 from 0.741 to 0.890 at IoU 0.25 at some cost in precision, and object-to-room ARI from 0.488 to 0.696 (p=0.002, ahead on every floor).

1 Introduction

Room grouping is pivotal because room-named navigation and object queries depend on it. ProClosure addresses the ambiguity of incomplete monocular-video boundaries by closing gaps according to width and assigning objects through visibility.

1 Introduction

  • Room-layer errors propagate to object groupings and queries, making room partitioning pivotal for navigation and language-based interaction [1], [2], [3].A room-named instruction such as searching the kitchen depends on the room layer.Source: §1 · p. 1
  • Monocular video records walls only where observed, so an unobserved wall and a doorway can produce geometrically indistinguishable gaps.Prior room-layer methods do not treat this boundary incompleteness as their central difficulty; some monocular scene-graph systems add inertial input [5] or learned room-transition priors [6].Source: §1 · p. 2
  • ProClosure treats every boundary break by width, sealing openings during progressive inward thickening and using visibility to keep rooms distinguishable for object placement.Camera-trajectory positions serve as seeds, replacing heuristic map sampling.Source: §1 · p. 2
  • The method formulates room detection as progressive closure with per-region freezing and compares visibility-agreement with surface-majority object assignment under incomplete boundaries.The evaluation treats object-to-room assignment as a clustering-agreement problem.Source: §1 · p. 2

2 Related Work

Prior scene-graph and room-partitioning methods generally rely on sufficiently complete geometry, while monocular reconstruction leaves ambiguous boundary gaps. ProClosure draws on progressive morphological scales but freezes each room at its own closure scale.

2.1 Hierarchical 3D Scene Graphs

  • Hierarchical scene graphs group objects into rooms and storeys; prior systems use depth, inertial data, or supplied poses, while recent RGB approaches remain sparse [1], [2], [3], [5], [6].HOV-SG [3] recovers storeys, rooms, and objects; Mono-Hydra [5] uses inertial support, while LEXI-SG [6] uses learned image features for room transitions.Source: §2.1 · pp. 2–3

2.2 Partitioning Indoor Space into Rooms

  • Classical clearance-, connectivity-, and learned room-partitioning methods treat boundary breaks as architectural apertures, an assumption that fails when monocular video misses walls.Doorway detectors can cut maps at detected openings, whereas ProClosure uses door frames only for visibility and still seals unobserved gaps.Source: §2.2 · p. 3

2.3 Morphological Segmentation and Scale

  • Unlike a single closing radius, global granulometry, or fixed-marker watershed, ProClosure extracts and freezes each region at its own closure scale while carrying witnesses through contraction.A single radius trades sealing wide openings against preserving small rooms.Source: §2.3 · p. 3

3 Methodology

The pipeline projects reconstructed structure into a top-down map, then detects rooms through progressive boundary closure and assigns objects using reconstructed surfaces or visibility agreement. Camera poses seed room witnesses, and each room is frozen when its pocket saturates.

3 Methodology

  • A monocular-video front end supplies a point cloud, trajectory, door frames, and object tracks; structural points are aligned and rasterised into a binary top-down map.The map uses scene-relative thresholds because the reconstruction has unknown absolute scale.Source: §3 · p. 4, §3.1 · p. 4

3.2 Room Detection by Progressive Boundary Closure

  • A room is a seeded free-space component declared when it saturates and then frozen, so each region closes at its own effective boundary-growth scale.This avoids imposing one radius across floors with different opening widths and room sizes.Source: §3.2 · pp. 4–5, §3.2.1 · p. 5

3.2.1 Progressive Closure

  • A pocket saturates when no cell remains far enough from both the boundary and existing seeds to host another seed; narrow openings close before the spaces they join.Very wide openings or spaces without a constriction can prevent separation.Source: §3.2.1 · p. 5

3.2.2 Seeds as Region Witnesses

  • Camera poses in the floor’s height band become seeds when they meet clearance and separation thresholds; deterministic placement avoids random map sampling.An undeclared seed threatened by boundary growth is relocated within free space, without crossing into a declared room.Source: §3.2.2 · p. 5

3.2.3 Rooms and Cleanup

  • Cleanup discards pockets closed by the raster edge, re-homes seeds that cannot see their assigned region, and uses detected door frames to block visibility through doorways.Door frames affect visibility tests, not the closure of the raster boundary.Source: §3.2.3 · pp. 6–7

3.2.4 Room Delineation

  • Room boundaries are delineated by seed sightline votes, then uncovered cells are filled through free space; this pass distributes area but cannot create or merge rooms.The fill is confined to the reconstructed footprint to avoid flooding the exterior of a walls-only map.Source: §3.2.4 · p. 7

3.3 Object Placement; 3.3.1 Surface Majority

  • Because centroid containment is brittle near uncertain boundaries, each object takes the room holding the largest share of its reconstructed surface points.Boundary cells are ignored; if all projected points lie on the boundary, cells are assigned to their nearest room through free space.Source: §3.3 · p. 8, §3.3.1 · p. 8

3.3.2 Visibility agreement

  • Visibility agreement compares object and room-seed sightline fans: fans within a room often cross, while few rays cross a narrow opening; assignment uses the highest qualifying room score.If no score reaches the threshold, the nearest free cell’s room is used as fallback.Source: §3.3.2 · p. 8

4 Experimental Setup

The quantitative evaluation uses HM3D-Semantics scenes with a controlled HOV-SG comparison on identical maps, while real handheld captures are qualitative. Metrics distinguish room partitioning from object-to-room assignment.

4.1 Datasets

  • Quantitative evaluation uses 6 HM3D-Semantics scenes across 10 floors, with posed RGB-D maps shared by ProClosure and HOV-SG; monocular reconstruction is evaluated separately.Handheld videos of three real-interior layouts are used only for qualitative results and failure analysis.Source: §4.1 · pp. 8–9

4.2 Evaluation Protocol and Metrics

  • Room partitioning is scored by two metric families, while object assignment uses Hungarian-matched accuracy, ARI, and NMI over 2,264 instances.ARI and NMI remain comparable when methods predict different room counts; room and object scores can disagree.Source: §4.2 · p. 9

4.3 Baselines and Comparison Methods

  • The baseline is a reimplementation of HOV-SG [3] scored on identical top-down maps, and internal ablations vary seed placement or object assignment while holding other stages fixed.The reimplementation uses a single grid, removing a misregistration source and strengthening the baseline.Source: §4.3 · p. 9

5 Results

Across the ten-floor comparison, object assignment favors ProClosure significantly, while room mIoU shows a positive but nonsignificant trend. The qualitative cases and discussion show that room-count fit, boundary fit, and object placement can diverge, with wide openings remaining a failure case.

5.1 Quantitative Comparison

  • Object accuracy was 0.801 against 0.684 and ARI was 0.696 against 0.488, with the differences attributed to room segmentation and significant at p = 0.002.The paired test found the methods separated on object assignment on all ten floors.Source: §5.1 · pp. 9–10

5.2 Qualitative Comparison

  • On 00829 floor 0, HOV-SG's mIoU was 0.842 against 0.795, yet our object accuracy was 0.847 and ARI was 0.752 despite loose room overlap.This example separates room-boundary fit from object-to-room agreement.Source: §5.2 · pp. 10–11
  • Across five rows, HOV-SG recovered 5, 3, 3, 2, and 5 rooms against 7, 7, 7, 4, and 7 regions; ours matched the count on two floors.The examples show HOV-SG under-segmenting, while ProClosure's room counts vary around the annotated counts.Source: §5.2 · p. 10

5.3 Ablation Study

  • The ablation study isolates seed placement, doorway sealing, and object-to-room assignment by changing one stage at a time and holding the others fixed.Seeding and doorway sealing are scored by room mIoU; assignment is scored by object ARI.Source: §5.3 · p. 12, Table 3 · p. 12

5.4 Limitations

  • On an open-plan floor, openings comparable to a room's extent caused spaces to seal together; seven of eleven annotated regions were absorbed, although object accuracy was 0.846 and ARI was 0.590.Only four regions were observed densely enough for room evaluation, and four rooms were recovered.Source: §5.4 · p. 12

5.5 Discussion

  • Room mIoU (p = 0.065) and trajectory seeding (p = 0.064) remain trends, and no controlled sweep tests per-region closure against a single global radius.The authors characterize ten floors as a small paired sample and limit claims to what the tests support.Source: §5.1 · p. 10, §5.5 · p. 12
  • Failure modes 1–3 affect all reported results, whereas modes 4–5 apply only to the monocular pipeline, not the quantitative evaluation with a supplied vertical and annotated objects.The quantitative evaluation uses dataset-provided vertical orientation and annotated instances.Source: §5.5 · p. 12

6 Conclusion

ProClosure recovers rooms from pose-free monocular video by closing boundary gaps at their own scales, then assigns objects to rooms using their reconstructed extent. Across ten floors, it improves object-to-room ARI over the clearance-and-flooding construction, while the paper notes limits for open-plan spaces.

6 Conclusion

  • ProClosure treats boundary breaks by width rather than cause, freezing each room as its free-space region closes and assigning objects by reconstructed extent.Objects near a boundary are decided by a narrowing margin rather than a test that flips; neither stage classifies openings as doors.Source: §6 · pp. 12–13
  • ARI reaches 0.696 against 0.488 for the clearance-and-flooding construction, with a significant difference over ten floors.The evaluation covers 10 floors of six HM3D-Semantics scenes and 2,264 annotated instances.Source: §6 · p. 13

A Reconstruction and Structural Segmentation

  • A pointmap SLAM front end and open-vocabulary segmentation produce structural points, door frames, object tracks, and a camera trajectory from monocular video.The structural cloud is projected into a top-down grid, while the monocular reconstruction is fixed only up to a global similarity.Source: A Reconstruction and Structural… · p. 15

C Properties of the Closure

  • Room count emerges from geometry as each seeded free-space pocket saturates, with distinct rooms closing at their own scales rather than one fixed radius.A narrow aperture and a wide opening can therefore close at different iterations; detected door frames affect visibility, not the partition criterion.Source: C Properties of the Closure · p. 15

D Rooms and Cleanup

  • Cleanup discards pockets bounded by the raster edge, resolves misplaced seeds, and delineates full room extents after freezing.Door-frame cells block visibility across open doorways, and seeds falling on them are removed rather than re-homed.Source: D Rooms and Cleanup · pp. 15, 17

B The Workflow in One Diagram

  • The workflow branches after seed groups form: surface majority uses dense room labels, whereas visibility agreement compares object sightlines with room-seed sightlines and falls back when agreement is zero.The first assigns each track to the room containing most of its reconstructed extent; the second does not require dense labels except for fallback.Source: Figure 5 · p. 16

E Object Placement Algorithm

  • Visibility agreement assigns a track using its sightline agreement with room seeds, and uses dense room labels only when no room reaches the agreement threshold.The alternative surface-majority rule assigns the track to the room containing most of its reconstructed extent.Source: Figure 5 · p. 16, E Object Placement Algorithm · p. 17

F Evaluation Protocol in Full

The evaluation uses two room-partition criteria with different prediction-to-region matching rules, and scores both on top-down grid cells. It also reports surface-point scores for comparability with prior work.

F Evaluation Protocol in Full

  • The overlap criterion takes independent maxima in each direction, so its pairing is neither one-to-one nor thresholded.Low prediction overlap indicates under-segmentation; low region overlap indicates over-segmentation.Source: F Evaluation Protocol in Full · p. 18
  • The instance criterion uses one-to-one Hungarian matching on IoU, counting a true positive when a matched pair exceeds IoU threshold 0.25 or 0.5.It reports precision, recall, F1, and mean IoU over matched pairs, and penalizes loose overlaps more strictly than the overlap criterion.Source: F Evaluation Protocol in Full · p. 18
  • Room criteria are computed over top-down grid cells, avoiding surface-area weighting and unlabeled ceiling or upper-wall samples.Surface-point figures are also reported alongside for comparability, and floor scores are averaged equally across ten floors.Source: F Evaluation Protocol in Full · p. 18

G Three Findings Across the Results

Across the results, the authors attribute much of the room-partitioning difference to predicted room counts and report a separable benefit from doorway sealing. They also walk through a floor example, review partitioning families, and describe the canonical-frame alignment.

G Three Findings Across the Results

  • The method predicts 74 rooms for 72 observed regions, versus HOV-SG’s 44, and doorway sealing improves all ten floors by +0.034 mIoU.The room-count difference largely drives Table 1; the doorway-seal stage uses an input the baseline does not receive.Source: G Three Findings Across the Results · p. 18

I Room-Partitioning Families

  • Clearance-based methods depend on clearance contracting at passages, connectivity-based methods sever narrowed free-space graphs, and learned methods regress room polygons from maps or point clouds.Learned approaches require annotated training data and, in most cases, near-rectilinear layouts.Source: I Room-Partitioning Families · p. 18

H Phase A/B Walkthrough

  • On the 00824-Dd4bFSTQ8gi floor, nine of 11 predicted rooms match ground truth at both thresholds, with F1 0.86 and mean matched-pair IoU 0.695.The walkthrough shows seeds, retained sightlines, delineated rooms, and matched regions.Source: H Phase A/B Walkthrough · p. 18, Figure 6 · p. 19

J Canonical Frame in Full

  • Canonical alignment fits a floor plane to orient the cloud, uses dominant wall direction to set yaw, and rasterises the lower half of the scene’s height envelope.The closure itself is isotropic and does not require a Manhattan layout.Source: J Canonical Frame in Full · p. 19

K Further Limitations

The paper identifies geometric cases where width-based closure cannot separate spaces, alongside limits from incomplete observation, alignment, and track fragmentation. It also details seed relocation, closure mechanics, table trade-offs, and ground-truth coverage.

K Further Limitations

  • A flush junction with no constriction cannot be split by width-based growth, and the top-down map lacks evidence such as ceiling-height or floor-material changes that could mark the boundary.On the cited floor, the adjoining spaces share a 262-cell frontier and are recovered together.Source: K Further Limitations · p. 19

L Seed Relocation in Full

  • Threatened seeds move to the nearest admissible cell along free space, with geodesic distance preventing a relocation across a wall.Declared regions are excluded from the relocation search, and displacement is capped.Source: L Seed Relocation in Full · p. 20

M Progressive Closure in Full

  • Progressive closure dilates the occupied boundary while leaving declared regions untouched, and declares a seeded pocket when it contains no qualifying cell.A cell qualifies only when it is sufficiently far from both the boundary and current seeds.Source: M Progressive Closure in Full · p. 20

N Reading Table 1

  • 44 rooms for 72 observed regions gives HOV-SG P@.25 = 0.98 and R@.25 = 0.60, while 74 against 72 gives ProClosure 0.31 more recall for 0.10 less precision at that threshold.ProClosure leads on F1 at both thresholds, but at the stricter threshold its precision is 0.81 against HOV-SG’s 0.91.Source: N Reading Table 1 · p. 20

O Ground-Truth Provenance

  • Room evaluation includes the 72 of 105 annotated regions with sufficient observed extent, while object scores use all 105 instances.Room extents are projected onto the same grid as the top-down map, and object membership comes from instance annotations.Source: O Ground-Truth Provenance · p. 20

P Recurring Failure Modes

  • Recurring failure modes include wide openings, junctions without constrictions, largely unobserved regions, incorrect vertical alignment, and fragmented object tracks.The paper reports that 33 of 105 annotated regions fall below the observation threshold and are excluded from room evaluation.Source: P Recurring Failure Modes · pp. 20–21

Q Ablations in Full

The ablations test camera-pose seeding, the doorway seal, and object-assignment rules, alongside corrections for characteristic closure errors. The doorway seal is the only ablation reported to separate its variants at this sample size.

Q Ablations in Full

  • Camera-pose seeding gives mIoU 0.802 against 0.781 for farthest-point sampling, a trend that does not reach significance (p = 0.064).It leads on nine floors, requires no eligibility pool or random selection, and makes segmentation deterministic.Source: Q Ablations in Full · p. 21
  • Door-frame boundary treatment raises mIoU from 0.768 to 0.802 across all ten floors (p = 0.002), moving the room count from 71 to 74 against 72 observed regions.It is the only ablation reported here to separate its variants, and the only one using an input the baseline does not receive.Source: Q Ablations in Full · p. 21
  • Surface majority gives object ARI 0.696 against 0.689 for visibility agreement, but the difference is not significant (p = 0.193).It leads on seven of ten floors and is adopted for requiring no visibility threshold and costing less; the ray rule costs nobjnseedn2 ray segment tests per floor.Source: Q Ablations in Full · p. 21

R The Three Corrections

  • Three corrections discard raster-edge regions, re-home seeds that cannot see their assigned region, and treat detected door frames as boundary during visibility measurements.Door frames come from SAM3 in the monocular pipeline and dataset annotations on HM3D.Source: R The Three Corrections · p. 21

S Implementation Details

The implementation reconstructs and segments a point cloud, projects structural points into a top-down map, and applies progressive closure and object assignment. The section also distinguishes controlled comparison conditions from published scores reported on different inputs and evaluation protocols.

S Implementation Details

  • MASt3R-SLAM supplies a keyframe trajectory and dense cloud, while SAM3 separates structural surfaces, door frames, and object tracks; structural points are rasterised into an upright, axis-aligned top-down grid.The BEV canvas is 1024 × 1024 at 0.05 m/px, then cropped to the observed extent; denoising uses a 3 × 3 opening and removes components under 50 px.Source: S Implementation Details · p. 21
  • Progressive closure uses shared settings across the three seeding variants, while camera-pose seeds use every recorded frame pose without a pool or cardinal test.The object-assignment variants were produced by a scoring harness outside the code snapshot, so its internal thresholds are not restated.Source: S Implementation Details · pp. 21–22
  • The method drops delineated regions under 40 px and rooms holding fewer than 2000 annotated 3D points at point labelling; interior fill is confined to a footprint eroded by 6 px.Delineation uses 50 rays per seed with a 150 px sightline cap.Source: S Implementation Details · p. 22

T Systems that Recover a Room Layer

  • Table 4 groups room-layer systems by required inputs; among the last two systems free of poses, depth, and calibration, only ours recovers rooms geometrically without a learned room-transition prior.Mono-Hydra is pose-free in the camera sense but requires IMU.Source: Table 4 · p. 22, T Systems that Recover a Room Layer · p. 22

U Published Numbers in Context

  • Published room scores are contextual, not a controlled comparison, because systems use different inputs, spatial units, and averaging schemes; the controlled comparison uses identical top-down maps and an identical harness.LEXI-SG is the nearest work by input assumption, but the authors could not run it directly.Source: U Published Numbers in Context · p. 22

V Scene-Wise Analysis

The per-floor analysis shows that the room-mIoU advantage varies across layouts rather than holding uniformly. The method leads on seven of ten floors, and the table reports a trend rather than a significant overall margin.

V Scene-Wise Analysis

  • Ours is ahead on 7 of 10 floors; Table 5 reports p = 0.065 for the paired Wilcoxon test, so the margin is a trend.The largest lead is on 00873 floor 0 (0.647 vs. 0.350), while HOV-SG leads on three floors, including 00829 floor 0, 00877 floor 1, and 00890 floor 0.Source: V Scene-Wise Analysis · p. 22, Table 5 · p. 23
  • Per-floor mIoU varies considerably for both methods: HOV-SG ranges from 0.350–0.936 and ours from 0.647–0.939.Neither mean-over-floors total is representative of every floor.Source: Figure 7 · p. 23

W Ablation Figures

The ablation figure presents seed placement and doorway sealing on room mIoU, and object assignment on object ARI. The accompanying runtime analysis reports slower room detection and notes that closure-scale distributions are not recorded.

W Ablation Figures

  • Figure 8 shows three ablations, scoring seed placement and doorway seal on room mIoU and assignment on object ARI, with final-method variants in orange.Each panel prints paired Wilcoxon p-values and win/loss counts over ten floors.Source: Figure 8 · p. 23, W Ablation Figures · p. 23

X.1 Closure Scale

  • Per-room closure iterations are not persisted, so no closure-scale distribution is reported.Figure 3 provides only a qualitative illustration of the closure sequence.Source: §X.1 · p. 24

X.2 Runtime

  • Room detection takes 6.6 ± 3.4 s per floor versus 0.3 ± 0.1 s for HOV-SG on the same map, roughly twenty times slower.The closure loop performs one dilation and Euclidean distance transform over the N × N grid per iteration, giving O(TN^2); runtime ranges from 3.2 s on the smallest floor to 13.4 s on the largest. Door-frame extraction adds 5.5 ± 1.9 s per floor, dominated on HM3D by annotated-mesh loading and sampling.Source: §X.2 · p. 24

Related papers

Loading 2609.12614v1…