PaperlayerView exact revision on arXiv

Source-linked AI summary

Monocular SLAM Supported Object Recognition

Sudeep Pillai, John Leonard

arXiv:1506.01732v1cs.ROcs.CV

TL;DR

Frame-by-frame object recognition does not use the repeated, multi-view observations available to robots. This paper combines monocular SLAM, multi-view proposals, and efficient feature encoding; on UW-RGBD, it reports 81.5 mAP versus the 61.7 mAP detector performance reported in.

  • Problem

    Traditional recognition datasets mostly use arbitrarily selected scenes or objects, unlike robots’ continuous streams observing the same objects from multiple viewpoints.

    Source: §I · p. 1

  • Method

    The system combines monocular SLAM-based multi-view object proposals with efficient feature encoding and aggregates recognition evidence across views.

    Source: §I · p. 1, A. Multi-view Object Proposals · p. 3

  • Results

    81.5 mAP, compared with the 61.7 mAP detector performance reported in.

    Source: §IV · p. 7

  • Takeaways & Limitations

    The monocular system is competitive with RGB-D object-recognition methods and has runtime nearly constant with the number of identifiable objects.

    Source: §V · p. 8

    Recognition depends on the monocular visual-SLAM backend, which failed to reconstruct the scene on a few occasions in the experiments.

    Source: §IV · p. 8

Abstract

from arXiv · show

In this work, we develop a monocular SLAM-aware object recognition system that is able to achieve considerably stronger recognition performance, as compared to classical object recognition systems that function on a frame-by-frame basis. By incorporating several key ideas including multi-view object proposals and efficient feature encoding methods, our proposed system is able to detect and robustly recognize objects in its environment using a single RGB camera in near-constant time. Through experiments, we illustrate the utility of using such a system to effectively detect and recognize objects, incorporating multiple object viewpoint detections into a unified prediction hypothesis. The performance of the proposed recognition system is evaluated on the UW RGB-D Dataset, showing strong recognition performance and scalable run-time performance compared to current state-of-the-art recognition systems.

I. Introduction

The introduction argues that frame-by-frame recognition misses opportunities to combine observations of the same objects across a robot’s continuous, multi-view stream. The paper proposes a monocular SLAM-aware system that uses map and camera-location information, multi-view object proposals, and efficient feature encoding.

I. Introduction

  • Traditional datasets mostly use arbitrarily selected scenes or objects, whereas robots observe the same objects repeatedly from changing viewpoints.The introduction motivates integrating detection evidence across views while simultaneously localizing and mapping the environment.Source: §I · p. 1
  • A SLAM-aware system uses an incrementally built map and camera location, unlike frame-by-frame SLAM-oblivious recognition.The proposed system uses an RGB camera as its only sensory input.Source: §I · p. 1
  • Figure 1 depicts object localization and recognition by aggregating detection evidence across multiple views.The white annotations are actual system predictions.Source: Figure 1 · p. 1
  • The system combines monocular SLAM, multi-view-consistent object proposals, and efficient feature encoding to recognize objects from an RGB camera stream.Its pipeline reconstructs a semi-dense scene, projects proposals into views, encodes them, and aggregates likelihoods across viewpoints.Source: §I · p. 1, Figure 2 · p. 2
  • The system is evaluated on the UW-RGBD dataset against state-of-the-art methods, single-view proposals, and for runtime scalability.The introduction says the experiments assess recognition, proposal performance, and runtime scalability.Source: §I · p. 2

II. Related Work

Related work surveys recognition, proposal, and encoding approaches, emphasizing trade-offs in localization, scalability, and multi-view operation.

Sliding window techniques and DPM

  • Sliding-window HOG and DPM detectors scan templates across scales for each object type, making runtime depend on the number of categories.Scaling such schemes can trade recognition performance for speed.Source: §II · p. 2

Dense sampling and feature encoding methods

  • BoVW scales well but weakly represents spatial relations, while VLAD and Fisher Vectors aggregate residuals and have outperformed traditional BoVW approaches.The paper uses VLAD as a trade-off between descriptive power and computation time.Source: §II · p. 2

Object Proposals

  • Category-independent proposals can achieve 95–99% detection rates at 0.7 IoU using about 1,000–5,000 candidate windows, but irrelevant windows remain difficult to classify.Proposals reduce the search space for sliding-window methods.Source: §II · pp. 2–3

Scalable Encoding with Object Proposals

  • Integral-histogram box encoding was reported to run 18× faster than brute-force BoVW for 30,000 proposals and performed strongly on PASCAL VOC and ImageNet.Its runtime is independent of proposal-window size.Source: §II · p. 3

Multi-view Object Detection

  • Prior multi-view work includes RGB-D voxel labeling and joint camera, scene-point, and object-label estimation, with the latter reported at 20 minutes per image pair.Other approaches raised concerns including limited instance scalability, RGB-D requirements, or reliance on 3D object models.Source: §II · p. 3

A. Multi-view Object Proposals

The method uses a semi-dense monocular SLAM reconstruction to generate object proposals across views, exploiting temporal consistency to reject inconsistent edges and separate near- and far-field objects. It partitions the reconstruction, projects candidate regions into camera views, and filters proposals before classification.

A. Multi-view Object Proposals

  • Multi-view proposals exploit spatio-temporal consistency in a semi-dense reconstruction built with ORB-SLAM [27] and depth filtering derived from; LSD-SLAM was avoided because it occasionally failed on benchmark wide-baseline motion.The reconstruction retains edges consistent across views, supporting rejection of inconsistent observations and separation of near- and far-field objects.Source: A. Multi-view Object Proposals · p. 3, Figure 3 · p. 4
  • The reconstruction is partitioned using spatial density and edge-color information at four density thresholds, each varied by a factor of two, to create over-segmented seeds and remove spurious points.The resulting point cloud is used to propose objects.Source: A. Multi-view Object Proposals · p. 3
  • Over-segmented regions are projected onto individual camera views as seeds for occlusion handling, refinement, and bounded candidate-window generation.The figure shows per-frame proposal probabilities aggregated across views to infer the most likely object label.Source: A. Multi-view Object Proposals · p. 3, Figure 3 · p. 4
  • Candidates smaller than 20x20 px, occluding candidates estimated by median depth, and overlapping candidates above an IoU threshold of 0.5 are culled before classification.These filters address small windows, mis-identification from occlusion, and redundant proposals, respectively.Source: A. Multi-view Object Proposals · p. 4

B. State-of-the-art Bag-of-Visual-Words with Object Proposals

The section describes object proposals with dense visual descriptors and a spatial-pyramid VLAD representation, then classifies them with a linear model. FLAIR makes descriptions of many boxes efficient through integral-histogram lookups.

B. State-of-the-art Bag-of-Visual-Words with Object Proposals

  • Dense 131-dimensional SIFT-plus-RGB samples are reduced by PCA to 80 dimensions, then quantized into a 64-word vocabulary.Source: B. State-of-the-art… · p. 4
  • VLAD is chosen over classical BoVW as more descriptive, and is reported to match Fisher-vector performance with slightly reduced computation time.Source: B. State-of-the-art… · p. 4
  • VLAD descriptions aggregate feature residuals from their quantized centers, use signed-square-rooting with α=0.5 and L2 normalization, then concatenate 1×1, 2×2, and 4×4 spatial bins.Source: B. State-of-the-art… · p. 5
  • FLAIR uses integral VLAD histograms for fast box descriptions, with computation independent of box area and only a few extra lookups for spatial-pyramid bins.Source: B. State-of-the-art… · p. 5, Figure 4 · p. 5
  • A linear classifier is trained with stochastic gradient descent; the method extends it to multiple classes one-versus-all and bootstraps wrongly classified negatives for two hard-negative-mining epochs.Source: B. State-of-the-art… · p. 5

C. Multi-view Object Recognition

The method uses semi-dense monocular SLAM maps and camera poses to propose object regions across views, then combines their class evidence. Its likelihood formulation sums per-view log-likelihoods under a conditional-independence assumption.

C. Multi-view Object Recognition

  • ORB-SLAM-based semi-dense mapping supplies a scale-ambiguous map, keyframes, and poses; map-derived object seeds are projected into frames as proposals.Source: C. Multi-view Object Recognition · p. 5
  • Median seed depths and a depth buffer support projecting non-occluding proposals; the detector estimates class probabilities per object cluster and image.Source: C. Multi-view Object Recognition · p. 5
  • Fig. 5 shows detections without cross-frame aggregation, whereas Fig. 6 depicts sequence-wide evidence aggregation for robust recognition.Source: Figure 5 · p. 6, Figure 6 · p. 6
  • Assuming view features are conditionally independent given the class, cluster class scores sum log-likelihoods across observable viewpoints.Source: C. Multi-view Object Recognition · p. 6

IV. Experiments

On the UW RGB-D dataset, SLAM-aware multi-view recognition performs comparably to an RGB-D method while improving on frame-based recognition and using fewer proposals than BING for similar or better recall. The system also offers near-constant runtime as the number of identifiable categories grows, though it depends on successful monocular SLAM reconstruction.

Experiment 1: Single-View Recognition

  • 81.5 mAP for the proposed per-frame system exceeded the 61.7 mAP reported for.Recognition was evaluated per image and averaged across test images.Source: §IV · p. 7

Experiment 1: Multi-View Recognition

  • 89.8 mAP on successfully reconstructed scenes for the five objects, compared with 90.9 mAP for HMP2D+3D.The comparison uses a monocular RGB system against a method that uses RGB and depth.Source: §IV · p. 7, Table I · p. 7

Experiment 2: Multi-View Objectness

  • Multi-view proposals achieve better recall than BING at a particular IoU threshold while using considerably fewer proposals across UW RGB-D scenes.Figure 8 varies BING's proposal count; numbers beside labels indicate average windows proposed per image.Source: §IV · p. 7, Figure 8 · p. 7

Experiment 3: Scalable Recognition and Run-Time Evaluation

  • For 51 categories, the system has near-constant runtime, while sliding-window approaches become impractical at ≥4 s; feature extraction and encoding take approximately 1.6s versus 5.15s per image for.The reported comparison uses a 4px step size for the proposed system and a 2px step size for.Source: Table II · p. 8, §IV · p. 8

Discussion and Future Work

  • The recognition system inherits monocular SLAM's robustness concerns, and the semi-dense vSLAM implementation sometimes failed to reconstruct a scene.The reported multi-view result covers scenes successfully reconstructed by the mapping system.Source: §IV · pp. 7–8

V. Conclusion

The paper presents a monocular SLAM-aware object-recognition system designed to provide robust, scalable recognition. Its evaluations show competitive performance, and the authors conclude that it may support recognition amid object clutter and viewpoint ambiguity.

V. Conclusion

  • The system combines semi-dense monocular SLAM for object proposals with efficient feature encoding for recognition.The proposed runtime is nearly constant with the number of identifiable object categories.Source: §V · p. 8
  • Evaluations show the monocular SLAM-aware solution is competitive with current state-of-the-art RGB-D object-recognition methods.The passage characterizes recognition as robust and scalable compared with classical SLAM-oblivious methods.Source: §V · p. 8
  • The authors suggest such a monocular system could help robots recognize and act on objects despite clutter and viewpoint-related ambiguity.Source: §V · p. 8
Loading 1506.01732v1…