PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Mapping of Underwater Litter Using Camera-Sonar Fusion

David Rete, Patrick Boros, Lucian Busoniu

arXiv:2609.39898v1cs.RO

TL;DR

Underwater litter surveys face camera and sonar limitations, and fixed paths may waste search effort. The paper fuses both sensors in an occupancy map and actively selects views; experiments report faster discovery than lawnmower coverage and better performance from fusion than either sensor alone.

  • Problem

    Underwater cameras suffer from turbidity and inconsistent lighting, sonar is often noisy and low-resolution, and fixed paths can reobserve confirmed areas while missing regions needing further observations.

    Source: §I · p. 1

  • Method

    The method uses Bayesian camera-sonar fusion in a shared litter occupancy map and selects next-best views by balancing exploration of uncertain regions against exploitation of likely litter.

    Source: §I · p. 1, §V · p. 6

  • Results

    Active search finds litter faster than lawnmower coverage, and sensor fusion is roughly twice as fast at finding all objects as either sonar or camera alone.

    Source: §IV · pp. 5–6, §V · p. 6

  • Takeaways & Limitations

    In the reported experiments, active mapping outperforms predefined fixed-pattern coverage, and combining complementary sensor modalities works better than either sensor separately.

    Source: §V · p. 6

    The segmentation model’s very good performance is expected for the simulated problem but is unrealistic for real underwater sensing, where environmental effects and sensor imperfections introduce variability.

    Source: A. Instance segmentation from sonar… · p. 3

Abstract

from arXiv · show

Marine litter is a growing threat to the underwater ecosystem, driving demand for autonomous survey methods that can locate debris efficiently over large areas. Existing survey methods typically follow predefined paths or operate with a single sensing modality, typically a camera (with image quality suffering in poor-visibility conditions) or sonar (usually noisy and low-resolution). We present an active mapping framework in which a forward-looking sonar and a camera both feed into a shared Bayesian occupancy map, and an optimization problem is solved at each step to decide on the next best view. Candidate viewpoints are scored by a two-term utility that balances exploration of uncertain regions via voxel entropy against exploitation of likely objects. Each sensor is characterized by range- and bearing-dependent detection and false-alarm probability tables determined from data. We evaluate the approach in a realistic underwater simulator, demonstrating that active mapping finds objects faster than a lawnmower coverage pattern, and that the dual-sensor approach works better than using either of the individual sensors.

I. INTRODUCTION

Underwater litter surveys contend with camera and sonar limitations, while fixed search paths can waste effort. The paper proposes active camera-sonar mapping that balances exploring uncertain seafloor with confirming likely litter.

I. INTRODUCTION

  • Cameras are impaired by turbidity and inconsistent lighting, while sonar is often noisier and lower-resolution, and fixed paths can reobserve confirmed areas while missing regions needing more observations.These constraints are especially problematic because litter can be small or visually ambiguous.Source: §I · p. 1
  • The proposed method fuses camera and sonar detections through Bayesian updates to maintain a litter occupancy map and select the next-best-view waypoint.Its mapping-progress objective balances uncertainty reduction against confirmation of likely litter.Source: §I · p. 1
  • Unlike classical mapping objectives that reduce overall uncertainty, the two-term objective separates exploration of uncertain seafloor voxels from exploitation of likely but unconfirmed litter.The authors argue that confirming litter matters more for this task than uniformly lowering uncertainty across the map.Source: §I · p. 1
  • Prior work includes aerial exploration planners balancing uncertainty reduction and regions of interest, as well as multitarget search methods that sometimes include exploration.The cited deep-reinforcement-learning extension is described as replanning faster than optimization-based planning.Source: §I · pp. 1–2
  • The stated novelty is closing the loop between camera-sonar fusion and active underwater target search, unlike underwater SLAM work focused heavily on reducing pose uncertainty.Related marine work addresses drifting surface targets or viewpoint planning for classifying already detected targets.Source: §I · p. 2

A. Occupancy-grid mapping

The map represents seafloor occupancy probabilistically at the voxel level. Bayesian updates incorporate uncertain sensor measurements, and voxel entropy expresses uncertainty in the estimated state.

A. Occupancy-grid mapping

  • The seafloor is represented as a 3D grid of voxels, each with a Bernoulli occupancy probability and an underlying binary occupied-or-not state.Bayesian updates use uncertain sensor measurements for voxels in view, starting from prior values that may be 0.5 when no prior information is available.Source: A. Occupancy-grid mapping · p. 2
  • Bayesian voxel updates depend on the sensor-measurement likelihoods for occupied and unoccupied states.The passage gives the update in terms of the prior occupancy probability and the measurement probabilities.Source: A. Occupancy-grid mapping · p. 2
  • Voxel entropy measures the uncertainty with which each voxel’s state is known.The implementation uses Octomap, an octree-based multi-resolution representation of the 3D environment.Source: A. Occupancy-grid mapping · p. 2

B. Deep learning for computer vision

The section distinguishes computer-vision tasks by their spatial output, then identifies Mask R-CNN instance segmentation as the model used in this paper.

B. Deep learning for computer vision

III. ACTIVE DUAL-SENSOR UNDERWATER MAPPING

Experiments use the MARUS underwater simulator and its MiniTortuga vehicle equipped with camera and sonar. The section introduces the active-search framework components.

III. ACTIVE DUAL-SENSOR UNDERWATER MAPPING

  • All reported experiments use MARUS, a Unity-based marine-robotics simulator with realistic sensor-image rendering and vehicle dynamics.Source: §III · p. 2
  • The simulated platform is a MiniTortuga UUV with a forward-looking camera and sonar, surveying static seafloor objects representing litter.The scene includes cubes and spheres.Source: §III · p. 2
  • The framework is organized around instance segmentation, sensor models, map updates, and an active-search planner.Source: §III · p. 2

A. Instance segmentation from sonar and camera images

The camera and sonar use separate Mask R-CNN instance-segmentation models trained on simulated sensor images. Test results show confident detections and near-ideal precision–recall behavior, while confidence is softened to better represent realistic sensing uncertainty.

A. Instance segmentation from sonar and camera images

  • The camera and sonar use separate Mask R-CNN models with a ResNet-50 backbone and Feature Pyramid Network, trained on synthetic MARUS datasets with varied object configurations.Ground-truth segmentation masks are polygonal outlines derived from sensor hit points.Source: A. Instance segmentation from sonar… · pp. 2–3
  • Test-set precision remains close to its maximum across most recall levels, with false positives appearing only at very low confidence thresholds for very high recall.The confidence score distributions are also described as highly concentrated near one.Source: A. Instance segmentation from sonar… · p. 3
  • Figure 3 shows camera and sonar detections with class bounding boxes and more precise segmentation masks behind them.The supplied figure caption identifies the outputs as instance segmentation results on test data.Source: A. Instance segmentation from sonar… · p. 3, Figure 3 · p. 3
  • Because simulated model performance is unrealistically strong for real underwater sensing, inference-time temperature scaling makes confidence scores more uniform to reflect greater uncertainty.The paper attributes real-world variability to environmental effects and sensor imperfections.Source: A. Instance segmentation from sonar… · p. 3

B. Sensor models

The sensor models estimate geometry-dependent detection and false-alarm probabilities from simulated camera and sonar measurements. They assume the UUV and sensor poses are sufficiently known, and use range and bearing to represent relative viewing geometry.

B. Sensor models

  • Sensor uncertainty is modeled with approximate true-positive and false-positive probabilities that depend on sensor pose and voxel center; for the horizontal, fixed-altitude UUV, relative geometry reduces to range and bearing.The framework assumes pose uncertainty can be disregarded.Source: B. Sensor models · p. 3
  • Target-present runs estimate true-positive probabilities, while empty-scene runs estimate false alarms from detections when the target is absent.The sweeps vary relative viewing geometry, and the same motion pattern is replayed for empty scenes to match measurement distributions.Source: B. Sensor models · pp. 3–4
  • A fixed confidence threshold of 0.8 converts each image into a binary detection, and empirical detection frequencies in geometry bins estimate the sensor probability tables.The tables are visualized as heatmaps of reliability across viewing geometries.Source: B. Sensor models · p. 4

C. Map and updates

The map assumes known seafloor geometry but unknown litter occupancy. Camera and sonar measurements update the shared voxel map sequentially when both arrive in the same cycle.

C. Map and updates

  • The seafloor geometry is known as a point cloud discretized into 0.2 m voxels, while which voxels contain litter remains unknown.The map’s goal is to determine occupancy from sensor readings.Source: C. Map and updates · p. 4
  • The sonar true-positive probabilities are estimated and presented in Fig. 5.Source: Figure 5 · p. 4
  • Asynchronous camera and sonar measurements are treated as conditionally independent given the map; when both arrive in a cycle, sonar updates the prior before camera updates the posterior.The same Bayesian update formula is applied for each sensor.Source: C. Map and updates · p. 4

D. Active search

Active search chooses a next waypoint to find objects faster than uniform coverage by balancing uncertainty reduction with refinement of likely object candidates.

D. Active search

  • At each active-search step, the planner enumerates candidate waypoints and chooses the one that maximizes the objective.The stated goal is to find objects faster than uniform coverage patterns such as lawnmower or spiral paths.Source: D. Active search · p. 4
  • The field of view ϕ(w) is the set of voxel centers within the sensors’ fields of view; with both sensors, it is their union.Source: D. Active search · p. 4
  • The exploration term prioritizes voxels with greater entropy, while the exploitation term targets voxels whose occupancy probabilities fall within a specified interval.The upper interval bound avoids repeatedly focusing on voxels already clearly part of an object.Source: D. Active search · pp. 4–5
  • The weight α trades off exploration and exploitation: smaller values favor covering unseen areas, while larger values favor refining object candidates.A waypoint specifies planar coordinates and UUV yaw; the vehicle navigates to the selected waypoint at fixed velocity while sampling sensors at their own frequencies.Source: D. Active search · p. 5

IV. RESULTS

In simulation, active search found objects faster than lawnmower coverage, and camera-sonar fusion outperformed either sensor alone. Its advantage persisted in a second object-cluster arrangement.

IV. RESULTS

  • In the first experiment, lawnmower coverage used 12 m strip spacing across three passes; active-search waypoints used a 6 m grid and eight heading angles.The simulated scene contained 11 objects in three clusters.Source: §IV · p. 5
  • For camera-only measurements in the second experiment, waypoint grid spacing was reduced to 3 m; the other parameters remained the same.Source: §IV · p. 5
  • By 27 m, active search found all 11/11 objects, versus 4 for lawnmower mapping; lawnmower mapping required 110 m to find all objects.Fig. 6 plots detected-object count against distance traveled.Source: Figure 6 · p. 5, §IV · p. 5
  • With active search, sensor fusion was roughly twice as fast at finding all objects as either sonar-only or camera-only mapping.There was no clear winner between the two single-sensor setups.Source: Figure 8 · p. 5, §IV · p. 6
  • Active search maintained its advantage in a second scenario with object clusters aligned at the top of the scene.Fig. 9 shows the comparison.Source: Figure 9 · p. 6, §IV · p. 6

V. CONCLUSIONS

The method fuses forward-looking sonar and camera observations into a shared probabilistic map for active underwater-litter search. In simulation, active mapping outperforms fixed-pattern coverage, and sensor fusion outperforms either modality alone; future work includes field trials and planner and sensor-model extensions.

V. CONCLUSIONS

  • Active mapping substantially outperforms predefined fixed-pattern coverage, while fused sensor modalities work better than either sensor separately in simulation.The shared probabilistic map uses range- and bearing-dependent sensor models, and viewpoint selection balances exploration with exploitation.Source: §V · p. 6
  • The method fuses forward-looking sonar and camera observations into a shared probabilistic map and selects views by balancing exploration and exploitation.Sensor models depend on object range and bearing relative to the sensor.Source: §V · p. 6
  • Future work includes modeling sonar-camera likelihood couplings, testing alternative objectives and non-myopic planners, and evaluating the pipeline in SeaClear2.0 field trials.Source: §V · p. 6
Loading 2609.39898v1…