Source-linked AI summary
HorizonNet for visual terrain navigation
Bertil Grelsson, Andreas Robinson, Michael Felsberg, Fahad Shahbaz Khan
TL;DR
The paper tackles accurate USV position estimation in coastal and archipelago settings when GPS is denied or failing. It extracts horizon information with two CNNs, registers the horizon against DEM data using a Fourier-domain MOSSE filter, and achieves GPS-level accuracy in field trials.
Problem
The paper addresses accurate USV position measurement in coastal or archipelago environments when GPS signals are denied or failing.
Method
Two CNNs estimate camera orientation and segment the horizon in a 360° panoramic image, which is warped and registered with DEM data using a Fourier-domain MOSSE correlation filter.
Results
2.47±1.26m average deviation from GPS measurements was obtained, and proposed-method errors were significantly lower than spatial registration errors averaging 3.1m.
Takeaways & Limitations
Horizon-line registration alone provided very accurate position estimates and was reported as potentially generalizable to UGVs and UAVs operating on land.
Abstract
from arXiv · showhide
This paper investigates the problem of position estimation of unmanned surface vessels (USVs) operating in coastal areas or in the archipelago. We propose a position estimation method where the horizon line is extracted in a 360 degree panoramic image around the USV. We design a CNN architecture to determine an approximate horizon line in the image and implicitly determine the camera orientation (the pitch and roll angles). The panoramic image is warped to compensate for the camera orientation and to generate an image from an approximately level camera. A second CNN architecture is designed to extract the pixelwise horizon line in the warped image. The extracted horizon line is correlated with digital elevation model (DEM) data in the Fourier domain using a MOSSE correlation filter. Finally, we determine the location of the maximum correlation score over the search area to estimate the position of the USV. Comprehensive experiments are performed in a field trial in the archipelago. Our approach provides promising results by achieving position estimates with GPS-level accuracy.
I. INTRODUCTION
The paper addresses USV position estimation when GPS is unreliable by using panoramic horizon information and DEM-based terrain navigation. Its two-CNN method and Fourier-domain registration achieve GPS-level accuracy in an archipelago field trial.
- GPS can be unreliable in hostile ocean scenarios, motivating alternative position sensors for safe USV navigation.
- Terrain navigation correlates measurements of surrounding terrain with a spatial database to provide position measurements without externally controlled signals and sensors.
- CNNs are employed to extract terrain information from USV images and generate accurate position measurements.
- The method uses a 360° panoramic camera to extract the horizon, whose complete angular profile supports robust cross-bearing against a map.
- Two CNNs estimate camera orientation and segment the horizon line, after which MOSSE registration correlates it with DEM data in the Fourier domain.
- 2.5±1.3 meters average global position accuracy relative to GPS ground truth demonstrates GPS-level position estimates in the archipelago field trial.
II. RELATED WORK
Prior work applied CNNs to horizon or sky segmentation and used spatial-domain horizon registration for localization. The paper identifies marine-image gaps and introduces Fourier-domain horizon registration as an unencountered approach.
- Previous CNN-based horizon and sky segmentation methods were evaluated on non-marine imagery, with no known public benchmark for marine images and USVs.
- Prior horizon-registration methods used spatial-domain profiles, gradient descent, grid search, least-squares matching, or feature-based localization.
- The paper reports no previous horizon-line registration with DEM data in the Fourier domain.
- MOSSE-based adaptive correlation filters perform Fourier-domain tracking faster and more robustly to target-appearance variation than earlier spatial-domain trackers.
III. CLASSICAL METHODS FOR POSITION ESTIMATION
The classical method registers an observed horizon against a DEM by compensating for camera orientation and searching for the best geometric alignment. It uses image-processing refinement after grid-search registration, but is too slow for real-time position estimation.
- Camera orientation: Canny edges and Hough voting estimate the panoramic camera’s pitch and roll before image warping.The horizon plane appears as an S-shaped curve in the cylindrical panoramic projection.
- Horizon registration: The warped image is compared with a DEM-derived geometric horizon using a distance function, whose summed projected values are minimized.The warping compensates for camera orientation and produces an approximately level view for registration.
- Ground-truth generation: The algorithm generates geometric horizon and water-line labels from GPS, compass heading, and DEM ray-tracing.For each viewing direction, the maximum elevation angle along a DEM ray defines the horizon point.
- Grid search: Registration searches pitch, roll, and heading on a grid, using 0.25° steps over ±2° for pitch and roll.The minimum-score rotation is then used to project the geometric horizon and water line onto the warped image.
- Refinement and limitation: A final Sobel-based tuning step adjusts horizon and water-line locations, but spatial registration takes more than 1 s per position grid point.The classical method is therefore prohibitively slow for real-time applications.
IV. CNN-BASED POSITION ESTIMATION METHOD
The method uses CNNs to estimate camera orientation and segment the horizon, then registers the segmented horizon with DEM data using a MOSSE correlation filter to estimate USV position.
- An end-to-end CNN position estimator failed because only land objects provide useful positioning information, while sky and sea appearance varies over time.
- HorizonFinder estimates pitch and roll indirectly through an approximate horizon line, replacing Canny detection and Hough voting.
- The network predicts cos θ, sin θ, cos ϕ, and sin ϕ rather than the attitude angles directly, using an L1 loss.
- Predicted pitch and roll angles warp the panoramic image into an approximately level view for subsequent horizon processing.
- HorizonSegmenter outputs the horizon line in the warped image, and successive predictions provide relative heading changes through phase correlation matching.
- The MOSSE filter correlates segmented horizon lines with DEM-derived geometric horizons in the Fourier domain, selecting the maximum score over an XY search grid.
A. Field trial
The field trial collected panoramic imagery and GPS measurements from a tele-operated USV operating in an archipelago.
- 20k images were captured at 10 fps during a 45 minute trial using a Ladybug3 camera mounted on a 4m USV.
- GPS measurements from a U-blox EVK-8 receiver provided vessel positions at 1 fps.
- The rear view, covering roughly 30◦, was occluded by other sensors on the vessel.
B. CNNs for horizon detection and segmentation
The CNNs detect the horizon effectively, but HorizonSegmenter smooths away fine-scale structures that may matter for registration-based position estimation.
- 40 image sequences were selected, with 35 used for training and 5 for testing; each sequence contained 100 consecutive images.
- HorizonFinder’s average test error relative to the generated ground truth was less than 0.1◦.
- HorizonSegmenter learned the general horizon shape but missed high-frequency variations such as single trees and other thin structures.
- The authors note that imperfect segmentation must be considered because the segmented horizon is used for registration and position estimation.
C. Position estimates
The proposed MOSSE-based horizon registration estimates USV positions more accurately and efficiently than spatial registration, while water-line registration performs worse for small USVs.
- Position estimation: The method estimates position by correlating segmented horizon lines from 10 consecutive images with geometric horizon lines across an 80m×80m search region.Scores are computed every 4m, followed by second-order polynomial interpolation for sub-grid resolution.
- Position estimation: 2.47±1.26m average deviation from GPS was achieved with the proposed method, supporting GPS-level position accuracy.The estimate averages the USV position during capture of the 10 images.
- Registration comparison: The Fourier-domain MOSSE registration significantly outperformed spatial registration, which produced a 3.1m average position error.A t-test at the 95% significance level found lower position errors for the proposed method.
- Registration comparison: Fourier registration is more robust to discrepancies between segmented and geometric horizon lines because lower-frequency content remains sufficient for accurate matching.Spatial registration is more distracted by missing high-frequency variations and imperfect DEM representation.
- Registration comparison: Restricting spatial registration to heading-angle search degraded its average position error to 6.5m, demonstrating sensitivity to angular errors.This shortcut assumed the pitch and roll estimates were correct.
- Water-line registration: Water-line registration produced a 3.0m average position error, inferior to horizon-only registration for cameras on small USVs.Low camera height and small viewing angles make water-line position sensitive to camera height and provide little variation unless near land.
VI. CONCLUSIONS
The paper addresses GPS-denied USV positioning with panoramic horizon extraction, CNN-based processing, and Fourier-domain MOSSE registration. Experiments show GPS-level accuracy, while horizon-only registration generalizes more promisingly than water-line registration for small USVs.
- Conclusions: The method estimates USV position from a 360° panoramic image by combining two CNNs, image warping, horizon and water-line extraction, and DEM correlation with a MOSSE filter.The maximum correlation score over the search area determines the position estimate.
- Conclusions: Fourier-domain horizon registration outperformed spatial registration in both position accuracy and computational speed.The conclusion identifies this as a central advantage of the proposed method.
- Conclusions: For a small USV with the camera approximately 1m above sea level, water-line registration was not robust and degraded horizon-only position estimates.The conclusion limits this observation to the evaluated small-USV camera configuration.
- Conclusions: Horizon-line registration alone provided very accurate position estimates and may generalize to UGVs and UAVs operating on land.The paper presents this as a suggested extension beyond the maritime setting.