PaperlayerView exact revision on arXiv

Source-linked AI summary

GlassGuard: Verified Glass Plane Mapping for Robot Navigation

Hanwen Guo, Zhengzhi Lin, Yusen Xie, Ji Zhang

arXiv:2610.02110v1cs.RO

TL;DR

GlassGuard targets the challenge of placing glass correctly in 3D maps, where visual appearance leaves metric depth ambiguous. It combines image masks, LiDAR structure, and depth-free geometric verification to construct and consolidate glass planes. Under matched pinhole inputs, it reports higher coverage than evaluated baselines alongside fewer false voxels per frame.

  • Problem

    Image localization alone does not determine the metric 3D placement of glass, whose sparse visual features make depth ambiguous.

    Source: §I · p. 1

  • Method

    GlassGuard builds metric plane candidates from visual detections and LiDAR structure, checks their orientations using depth-free projective geometry, and consolidates accepted planes globally.

    Source: §I · p. 1

  • Results

    82.1% Ever coverage for GG-pin versus 61.0% for MonoGlass3D and 44.4% for GlassRecon; GG-pin has 16.7 false voxels per frame versus 85.4 for GlassRecon and 290.2 for MonoGlass3D.

    Source: C. Quantitative Analysis · p. 6

  • Takeaways & Limitations

    The resulting map is designed to cover physical glass while limiting false occupancy in traversable space.

    Source: §I · p. 1

    The current method assumes planar, near-vertical glass; faster detection and inference would raise its update rate.

    Source: §VI · p. 8

Abstract

from arXiv · show

Transparent and specular surfaces pose a serious challenge to LiDAR-based SLAM and navigation because laser returns may pass through glass, leaving collision boundaries absent from the map. Prior work attempts to reconstruct the missing surfaces, but inaccurate obstacle placement can create the opposite failure: contamination of traversable free space. Recognizing this dual requirement, we present GlassGuard, a navigation-oriented framework for reconstructing planar architectural glass from complementary visual and LiDAR evidence. We formulate success in terms of both glass coverage and free-space contamination and apply this principle throughout proposal verification and global map construction. A foundation vision model provides glass-instance masks, structural 3D cues generate metric plane hypotheses, and depth-free 2D projective geometry checks their orientations before they enter a consolidated global map. We evaluate GlassGuard in nine building-scale scenes spanning diverse glass structures, spatial scales, and lighting conditions, with more than one hour and 2.1 km of real-world robot traversal. GlassGuard achieves 85% of total glass coverage for its panoramic version. Under identical pinhole inputs, GlassGuard achieves 82% total coverage, compared with at most 61% for the evaluated baselines, while producing 5-17x fewer false voxels per frame. Qualitative examples with a navigation planner illustrate the reconstructed planes blocking paths through glass while leaving traversable routes open. The project page is available at https://glassguardproject.github.io/.

I. INTRODUCTION

GlassGuard addresses the challenge of placing glass surfaces correctly in 3D maps, combining visual glass detection with LiDAR-derived structure and geometric verification. Prior work offers image localization and cue-based or reconstruction methods, but metric placement and avoiding false occupancy remain central concerns.

I. INTRODUCTION

  • 905 nm compact LiDAR returns from glass are highly variable, potentially leaving maps incomplete or corrupted and compromising scene reconstruction, exploration, and safe navigation,,.Glass may transmit or specularly reflect laser energy, or return it only weakly.Source: §I · p. 1
  • Image-based glass localization does not resolve metric 3D placement: glass has few visual features of its own, making depth ambiguous and misplaced planes capable of occupying traversable space.Pipeline methods anchor detections to nearby metric measurements, but may associate glass with an occluder or structure behind it.Source: §I · p. 1
  • GlassGuard detects glass in 2D, builds metric plane candidates from surrounding LiDAR structure, verifies their orientations with depth-free projective geometry, and consolidates accepted planes globally.Its map is designed to cover physical glass while limiting false occupancy in traversable space.Source: §I · p. 1

II. RELATED WORK

  • Physical-cue methods use returns, reflections, additional sensing, or contact,,, [7],, [12], while visual methods localize glass in images but do not alone provide its metric 3D map location.The paper considers settings where reliable measurement cues and additional hardware cannot be assumed available.Source: §II · pp. 1–2
  • The evaluated visual-to-3D baselines place surfaces from a single frame without later checking or revising them; GlassGuard instead verifies and updates reconstructed glass persistently, [26],.MonoGlass3D is the representative learned baseline, and GlassRecon is the representative pipeline-based baseline.Source: §II · p. 2

III. PROBLEM FORMULATION

The formulation treats glass mapping as a two-sided occupancy problem: physical glass must be marked occupied, while traversable space must remain free. GlassGuard represents reconstructed glass as bounded planar segments inserted into the navigation map.

III. PROBLEM FORMULATION

  • A correct glass-aware map marks voxels intersecting physical glass as occupied and voxels containing no physical surface as free.The formulation evaluates correctness at voxel resolution ρ.Source: §III · p. 2
  • Spill is free space wrongly marked occupied; it captures the usability cost of false glass planes blocking navigable space.The figure illustrates false blockage from slanted placement, incorrect depth, and reflection-induced slant.Source: §III · p. 2, Figure 1 · p. 2
  • GlassGuard represents reconstructed glass as bounded planar segments whose union forms a surface, and inserts the voxels intersected by that surface into the navigation map..Source: §III · p. 2

IV. METHOD

The method combines glass-instance detection with structural LiDAR cues to generate bounded 3D plane candidates, then verifies and manages those hypotheses before mapping them. Its Slim SAM3 student retains much of the teacher’s reported segmentation performance while reducing deployment memory.

A. Overview

  • GlassGuard detects instance masks, associates them with LiDAR geometry to obtain seeds, constructs bounded plane candidates, verifies orientations, and consolidates hypotheses for the navigation map.Global management merges or absorbs overlapping hypotheses using accumulated seed support.Source: A. Overview · p. 2, Figure 2 · p. 3

B. Reliable 2D Instance Detection: Slim SAM3

  • Confidence-guided structural pruning removes low-importance MLP channels and attention heads, and the original SAM3 briefly distills the slim student.Pruning importance is based on the model’s glass confidence, without manual pruning labels.Source: B. Reliable 2D Instance Detection:… · pp. 2–3
  • IoU is 0.748/0.671 for the student versus 0.815/0.713 for SAM3 on GDD/GSD-S, respectively.The table reports glass-instance segmentation on GDD and GSD-S.Source: Table I · p. 3, B. Reliable 2D Instance Detection:… · p. 3
  • GSD-S recall is 0.918 for the student, the table’s best, while GDD recall decreases from 0.895 to 0.826; quantization reduces reserved VRAM by approximately one third.Recall is the metric the pipeline depends on; the VRAM comparison is against the original SAM3 deployment.Source: B. Reliable 2D Instance Detection:… · p. 3

C. Associate 2D to 3D: Pillar Construction

GlassGuard builds metric glass-plane candidates from structural cues associated with LiDAR seeds, then checks their orientation against a depth-free image reference. The accepted candidate supplies metric placement and pane extent.

C. Associate 2D to 3D: Pillar Construction

  • Structural pillars provide more stable plane evidence than direct fitting to sparse LiDAR returns, with vertical pairs defining a direction and horizontal pillars supplying theirs from their axis.Vertical pillars are cells occupied across consecutive height layers; horizontal pillars are dense seed clusters with a strong line-like horizontal PCA response.Source: C. Associate 2D to 3D: Pillar… · p. 3

D. Ray-Cast Orientation Verification

  • A reliable mask-boundary silhouette line defines an interpretation plane for possible 3D lines, and two image lines for a shared in-plane direction resolve that ambiguity.Two independent direction families yield the plane normal; for architectural glass, gravity can replace the second family, allowing one sufficiently conditioned horizontal edge to provide a yaw reference.Source: D. Ray-Cast Orientation Verification · pp. 3–4
  • The image-derived yaw reference is compared with the LiDAR candidate’s yaw, with angular disagreement indicating a false blockage whose lateral displacement grows with distance from the pivot.For pane length L and spatial tolerance τ, the characteristic angular tolerance scales as arcsin(2τ/L), so longer panes require tighter agreement; the fixed τ is 0.5 m.Source: D. Ray-Cast Orientation Verification · p. 4, Figure 3 · p. 4
  • The orientation test checks yaw consistency, not candidate depth or metric placement.After acceptance, support rays intersect the candidate’s metric plane to determine observed pane endpoints and bounded extent.Source: D. Ray-Cast Orientation Verification · p. 4

E. Global Management

GlassGuard maintains accepted planes as global hypotheses that can be reinforced, merged, or removed as new evidence arrives. Global management combines plane merging, seed–floor evidence, and multi-view verification.

E. Global Management

  • Global management merges repeated estimates of the same pane to avoid clusters of overlapping, misaligned planes, retaining the better-supported competitor and absorbing the weaker estimate’s extent.Tracks compete under boundary-ray wedge and ground-segment overlap criteria; accumulated LiDAR seed count is the primary support score, while vertically separated tracks do not compete.Source: E. Global Management · pp. 4–5
  • Seed–floor evidence removes a plane when its ground-grid voxel is dominated by floor points, indicating a walkable region.Competing floor and glass-seed counts are updated in a 0.1 m grid and clamped to [0, 3]; a single floor observation is insufficient because real glass may reach the ground.Source: E. Global Management · p. 5
  • Multi-view verification evicts a plane when its projection increasingly falls outside detected glass masks, addressing incorrect-depth planes that can pass a single-view yaw check.Only well-observed tracks are judged; rising off-mask evidence accumulates toward eviction, while an off-mask fraction of at least 0.55 triggers immediate eviction.Source: E. Global Management · p. 5
  • Each cycle, current tracks are rendered as wall points for terrain analysis and planning, so placed planes block paths beyond the camera view and removed planes leave the collision map on the next cycle.The planner itself stores no persistent glass map; its input changes as the glass map evolves.Source: E. Global Management · p. 5

A. Real-World Setup

The evaluation replays timestamped predictions from nine real-world scenes against shared manually annotated ground truth. Methods receive matched LiDAR and RGB inputs, with GG-360 additionally using a full panoramic image.

A. Real-World Setup

  • The robot carried a 10 Hz Livox Mid-360 LiDAR and a 1920×640 Ricoh Theta panoramic camera, and GlassGuard averaged approximately 0.74 s per frame on an RTX 5070 Ti laptop GPU.Its planes were inserted into a LiDAR autonomy stack providing state estimation, terrain analysis, local planning, and TARE exploration.Source: A. Real-World Setup · p. 5
  • All LiDAR-using methods received the same 5 s stack of registered point clouds and synchronized RGB, while pinhole methods shared a pinhole RGB view and GG-360 received the full 360° image.This setup matched the input views within each comparison.Source: A. Real-World Setup · p. 5
  • Nine scenes across six indoor and outdoor environments covered varied glass structures and day, night, sunlight, and reflections, totaling more than one hour and 2.1 km of traversal.Planes were placed within 10 m of the robot, or 20 m in the two large outdoor scenes; timestamped predictions were replayed on recorded rosbags and evaluated against the same annotated ground truth.Source: A. Real-World Setup · p. 5

B. Evaluation Metrics

The evaluation measures glass coverage and false occupancy on a shared world-frame voxel grid. It distinguishes first detection from frame-by-frame map coverage and retention, and reports spill into ground and air.

B. Evaluation Metrics

  • Warning-distance coverage counts eligible, visible ground-truth glass voxels detected by distance d, with pinhole methods additionally requiring camera-frustum eligibility.Ground-truth glass beyond sensing range is excluded, and ever coverage counts detection in any eligible frame regardless of distance.Source: B. Evaluation Metrics · p. 5
  • Current-map coverage measures nearby glass represented at each frame, while retention measures whether correctly detected glass remains represented in later frames.A voxel counts when its cell or an adjacent cell is occupied; delayed detection or later removal reduces current-map coverage, and retention evaluates frames after the first correct coverage within 10 m.Source: B. Evaluation Metrics · p. 5
  • Spill counts predicted voxels containing neither active ground-truth glass nor an observed physical surface, separating ground spill from air spill.Spill is charged within 10 m of the false voxel; spill/frame is the mean count per frame, while Air and Ground report distinct accumulated false voxels and median distance to the nearest glass surface.Source: B. Evaluation Metrics · pp. 5–6
  • Figure 4 illustrates planner consequences across environments: GlassGuard variants recover glass barriers while largely preserving traversable space, unlike examples with baseline false occupancy or gaps.Its panels show RGB, Glass Blind, GG-360, GG-pin, MonoGlass3D, and GlassRecon; local and global paths are distinguished from predicted glass.Source: Figure 4 · p. 6

C. Quantitative Analysis

GlassGuard improves glass coverage while reducing false occupancy, and its panoramic variant raises coverage further. The planner-facing map retains high collision-band coverage, while finer grids reduce coverage across methods.

C. Quantitative Analysis

  • GG-pin averages 0.74 s per frame, compared with 3.6 s for GlassRecon, about 5× faster.Both methods use pinhole RGB and LiDAR.Source: C. Quantitative Analysis · p. 6
  • 82.1% Ever coverage for GG-pin exceeds 61.0% for MonoGlass3D and 44.4% for GlassRecon; GG-360 reaches 85.2%.The nine-scene occupancy comparison is at 1.0 m resolution.Source: C. Quantitative Analysis · p. 6, Table III · p. 7
  • 16.7 false voxels per frame for GG-pin compares with 85.4 for GlassRecon and 290.2 for MonoGlass3D; GG-360 carries 15.3.Under the same pinhole input, baselines maintain 5–17× more concurrent false occupancy than GG-pin.Source: C. Quantitative Analysis · p. 6, Table III · p. 7
  • At 0–2 m and 2–5 m, GG-360 covers 89.1% and 91.5% of demanded voxel-frames, respectively; GG-pin reaches 84.4% and 89.0%.At 0.5 m resolution, every method loses 7–19 points, while GG-360 remains 17 points ahead of GlassRecon at 0–2 m.Source: C. Quantitative Analysis · pp. 6–7, Table IV · p. 7

D. Component Ablation

Component ablations show that verification mechanisms chiefly distinguish configurations through false occupancy, with tradeoffs across spill and coverage measures. The mechanisms also complement one another, and GG-pin retains an advantage over baselines without global management.

D. Component Ablation

  • Ever coverage stays about 82–88% across ablation configurations, so the components distinguish themselves more on false occupancy than on coverage.Table V removes one component at a time from the full system.Source: D. Component Ablation · p. 7
  • Without the angle gate, concurrent spill rises by 75% and distinct false voxels rise by 126%; accumulated spill increases on every scene (+40% to +291%).Removing the entire manager raises accumulated spill by 78%, while Ever coverage differs by 0.9 points, within scene-to-scene spread.Source: D. Component Ablation · p. 7
  • Multi-view verification reduces accumulated spill by 10% and raises Ever coverage by 3.2 points, but lowers near-field current coverage by 5.4 points and raises concurrent spill from 18.7 to 19.4 voxels per frame.Planes incorrectly placed in depth that do not touch the ground require this mechanism for removal; no other component reaches them.Source: D. Component Ablation · p. 7
  • Removing merging, floor evidence, and multi-view verification individually changes concurrent spill by +74%, +33%, and −4%, about +104% total, whereas removing them together raises it by 352%.When one mechanism is absent, the others catch much of what it would have removed.Source: D. Component Ablation · p. 7
  • Without global management, GG-pin reaches 88.2% Ever coverage versus 48.0% and 51.8% for the baselines under the same pinhole input and insert-only policy.Its concurrent false occupancy is 87.7 versus 79.2 voxels per frame for GlassRecon, and below one third of MonoGlass3D’s; it admits 478 distinct false voxels versus 585 and 2059.Source: D. Component Ablation · p. 7

E. Qualitative Analysis

Qualitative planner examples show GlassGuard recovering glass barriers while largely preserving traversable space; the conclusion reports stronger coverage with less false occupancy than evaluated baselines. The authors identify faster inference and the planar, near-vertical assumption as areas for future work.

E. Qualitative Analysis

  • Without glass detection, planned paths cross physical glass; both GlassGuard variants recover barriers while largely preserving traversable space.Baseline false occupancy extends into traversable regions and can suppress or shorten displayed local paths; these examples illustrate planner integration, not quantitative navigation success.Source: E. Qualitative Analysis · p. 7

VI. CONCLUSION

  • Real-world experiments across diverse environments show higher glass coverage with substantially less false occupancy than the evaluated baselines.The authors identify faster detection and inference, and extension beyond planar, near-vertical glass to curved or strongly tilted glass, as future work.Source: §VI · p. 8
Loading 2610.02110v1…