Source-linked AI summary
Vision-based Real-Time Aerial Object Localization and Tracking for UAV Sensing System
Yuanwei Wu, Yao Sui, Guanghui Wang
TL;DR
Vision-based UAV sense-and-avoid needs automatic obstacle detection and tracking, while conventional trackers require manual initialization and salient-object methods lack temporal context. The paper combines saliency-map localization with Kalman prediction and refinement; across 15 sequences, it reports outperforming seven competing trackers, while noting difficulty with occlusion.
Problem
Vision-based UAV sense-and-avoid needs automatic obstacle detection and tracking, but conventional trackers require manual first-frame initialization and salient-object methods lack contextual information between frames.
Source: §I · p. 1Method
The method uses background-connectivity saliency maps to localize the object and combines a Kalman filter's coarse prediction with local detection and refinement.
Source: §I · p. 1, 1) Object localization · p. 4Results
Across 15 sequences, FOLT outperforms all seven competing trackers; one-pass CLE is 14.5 pixels versus CCT's 23.2 pixels, and precision is 0.83 versus 0.66.
Source: B. Comparison with the… · p. 6Takeaways & Limitations
The approach is presented for UAV navigation and is reported to initialize automatically, run fastest, and perform better than competing trackers.
Source: §V · p. 8The tracker handles occlusion poorly, though it can automatically re-localize the salient object when it re-appears in view.
Source: §V · p. 8, D. Limitation · p. 8
Abstract
from arXiv · showhide
The paper focuses on the problem of vision-based obstacle detection and tracking for unmanned aerial vehicle navigation. A real-time object localization and tracking strategy from monocular image sequences is developed by effectively integrating the object detection and tracking into a dynamic Kalman model. At the detection stage, the object of interest is automatically detected and localized from a saliency map computed via the image background connectivity cue at each frame; at the tracking stage, a Kalman filter is employed to provide a coarse prediction of the object state, which is further refined via a local detector incorporating the saliency map and the temporal information between two consecutive frames. Compared to existing methods, the proposed approach does not require any manual initialization for tracking, runs much faster than the state-of-the-art trackers of its kind, and achieves competitive tracking performance on a large number of image sequences. Extensive experiments demonstrate the effectiveness and superior performance of the proposed approach.
I. INTRODUCTION
Vision-based sense-and-avoid requires automatic obstacle detection and tracking, but existing tracking and detection approaches have limitations in initialization, temporal context, computation, or training data. The paper combines saliency-based detection with Kalman prediction and refinement, reporting strong performance against competing trackers.
I. INTRODUCTION
- Conventional trackers require manual first-frame initialization, are sensitive to initialization errors, and may return useless information after tracking failure.These issues complicate automatic obstacle tracking for UAV sense-and-avoid.Source: §I · p. 1
- Detector-based automatic initialization can require large training sets, while an optical-flow saliency tracker is computationally intensive and deep methods also need many samples.The object of interest may not be known in advance.Source: §I · p. 1
- The approach combines salient-object detection with Kalman prediction and detector-based refinement to estimate the target state recursively.The paper describes the process as prediction, refining, and correction.Source: §I · p. 1
- On a challenging sequence, the proposed method provides a high-quality saliency map and accurate target scale and position, unlike the cited detector and trackers.The sequence includes scale variation, out-of-view, and re-appearance challenges.Source: §I · p. 2
- The authors report better performance than state-of-the-art trackers in extensive experimental evaluations.The paper lists integrating target-specific saliency into a dynamic model and automatic initialization for UAV sensing among its contributions.Source: §I · p. 2
II. RELATED WORKS
Related work covers salient-object detection and visual tracking, while the proposed approach combines them in a recursive Kalman-filter framework. Its localization uses background connectivity and a prediction-constrained search region.
II. RELATED WORKS
- Salient-object detection computes a saliency map and segments an object's boundary, using cues such as contrast, region properties, or boundary connectivity.The cited methods are described as achieving pixel-level resolution.Source: §II · p. 2
- Visual-tracking research includes generative and discriminative approaches, with correlation-filter methods receiving particular attention for their speed.The reviewed trackers address challenges including illumination and scale variation, rotation, occlusion, and background clutter.Source: §II · p. 2
- On a challenging sequence, the proposed approach provides a high-quality saliency map and accurate target scale and position, while the compared methods have stated detection or tracking failures.The comparison includes scale variation, out-of-view, and re-appearance.Source: Figure 1 · p. 2
III. THE PROPOSED APPROACH
- FOLT estimates a salient object's state through recursive prediction, object detection, and correction in a robust Kalman-filter framework.A constant-velocity linear dynamic model represents motion, and the first frame is initialized using a saliency map over the entire image.Source: §III · p. 3
A. Dynamic model formulation
The dynamic model represents the salient object with a motion state and estimates its next-frame state through a linear stochastic transition. Under a Markov assumption and Gaussian model assumptions, the estimation is expressed in a standard Kalman-filter framework.
A. Dynamic model formulation
- The motion state contains the target center coordinates, velocities, width, and height, and evolves through a linear stochastic model.The transition model includes prediction noise; the paper states that Gaussian transition and observation models permit standard Kalman-filter estimation.Source: A. Dynamic model formulation · p. 3
- The measurement model specifies a measurement matrix and Gaussian observation noise with covariance R.The text also identifies diagonal elements of Q and R as representing covariance of size and position.Source: A. Dynamic model formulation · p. 3
- The model selects the current motion state using the observation and previously obtained states, then simplifies the objective with a Markov assumption.Under that assumption, the current state depends only on the previous state.Source: A. Dynamic model formulation · p. 3
B. Object tracking
Tracking localizes the object in a search region and infers its state from a post-processed saliency map. Localization updates the map using a background-connectivity cue within a region seeded by the Kalman prediction.
B. Object tracking
- Tracking combines localization in a search region with post-processing of the saliency map to infer the target state.The localization procedure applies a threshold and dilation to produce a tracking bounding box.Source: B. Object tracking · p. 3
1) Object localization:
- The predicted bounding box is expanded by a percentage to define a search region, where minimum-barrier distance from background seed pixels updates the saliency map and estimates position and scale.Pixels outside the search region retain their previous saliency-map values.Source: 1) Object localization · p. 4
- The background-connectivity cue assumes background regions connect to boundary seed pixels; path cost is the difference between maximum and minimum intensity along the path.The method considers four-adjacent neighbors and obtains saliency by minimizing path cost.Source: 1) Object localization · p. 4
- Raster and inverse-raster passes update pixel values in opposite directions, alternating for three passes in the described implementation.Each update uses neighboring pixels in the search region.Source: 1) Object localization · p. 4, Figure 3 · p. 4
2) Post-processing:
Post-processing thresholds the saliency map into a binary image and dilates it to segment the salient object. The algorithm takes the image, saliency map, and pass count as input and outputs the target state.
2) Post-processing:
- Thresholding converts the saliency map to a binary image, and dilation is then applied to segment the salient object in each frame.The text notes that global thresholding is inefficient under non-uniform illumination and lighting conditions.Source: 2) Post-processing · p. 4
- The local threshold uses the mean pixel value in a neighboring block centered at (a, b), with block size α × α and a nonnegative offset λ.Source: 2) Post-processing · p. 4
- Algorithm 1 takes image I_t, saliency map D, and pass count N as input, and outputs target state ŝ_t.Source: 2) Post-processing · p. 5
2 Keep the values in D outside the search region
For each frame, the algorithm predicts with a Kalman filter, tracks the object through raster scanning, and updates the measurement and correction steps.
2 Keep the values in D outside the search region
17 Compute the entire D every 10 frames
The localization process thresholds and dilates the saliency map, then uses the extracted region's minimum bounding box as the target-state estimate. In the recursive tracking loop, Kalman prediction, measurement, and correction support automatic detection and tracking, with the full saliency map refreshed every 10 frames.
17 Compute the entire D every 10 frames
- Dilation enhances the thresholded image, and the structuring element S defines the dilation operation.Source: §17 · p. 5
- After dilation, the minimum bounding box of the extracted region gives the target-state estimate.Examples are shown for the motorcycle 011 and airplane 016 sequences.Source: §17 · p. 5
C. Fast object localization tracking
- During correction, the posterior error covariance and Kalman gain are updated; the process also includes a measurement update and posterior covariance computation.Source: C. Fast object localization tracking · p. 5
- The recursive prediction, localization, and correction steps automatically detect and track salient objects in an image sequence.Source: C. Fast object localization tracking · p. 5
- The sequence attributes used for tracking characterization include scale and illumination variation, background interference, and in-plane and out-of-plane rotation.Source: Table I · p. 5
17 Compute the entire D every 10 frames
- The entire saliency map D is updated every 10 frames as a trade-off between accuracy and speed.Source: C. Fast object localization tracking · p. 5
IV. EXPERIMENTAL EVALUATIONS
The evaluation tests the tracker on a varied set of image sequences and compares it with seven state-of-the-art trackers. It uses one-pass and temporal-robustness evaluations; Table II reports quantitative evaluations across 15 sequences.
IV. EXPERIMENTAL EVALUATIONS
- The dataset contains over 6,700 frames, spans frame sizes from 320 × 240 to 1280 × 720, and includes challenges such as scale variation, occlusion, rotation, illumination, and background interference.Source: §IV · p. 5
- Competing trackers are initialized with ground truth in the first frame, whereas the proposed tracker runs automatically from the first frame through the end.Source: §IV · pp. 5–6
- The proposed tracker is quantitatively evaluated against seven competing trackers on 15 image sequences, with the best and second-best results marked in Table II.Source: §IV · p. 5, Table II · p. 6
- The evaluation uses one-pass evaluation and temporal-robustness evaluation, with temporal randomization as in reference.Source: §IV · p. 6
A. Speed performance
The tracker runs substantially faster than the compared methods and outperforms seven competing trackers on the reported tracking measures. Results cover 15 sequences under one-pass and temporal-robustness evaluations.
A. Speed performance
- At 141 fps, the tracker matched KCF’s speed while adapting its tracking box to target scale changes; it was also faster than the other listed trackers.It was more than ten times faster than CT and SAMF, five times faster than DSST and CCT, and about twice as fast as STC and CN.Source: A. Speed performance · p. 6
- For salient-object detection, the method ran at 149 fps versus MB+ at 49 fps, and the paper reports better detection performance than MB+.The reported 149 fps is three times MB+’s speed.Source: A. Speed performance · p. 6
B. Comparison with the state-of-the-art trackers
- The evaluation reports precision, centre location error (CLE), and success rate (SR), with precision using a 20-pixel CLE threshold and SR normally using an overlap threshold of 0.5.CLE is the Euclidean distance between the centers of the tracked and ground-truth boxes.Source: B. Comparison with the… · p. 6
- Figure 5 plots average precision and success rate over 15 sequences for OPE and TRE, while Table III reports sequence-level precision for the proposed and seven competing trackers.The Table III caption says the best and second-best results are marked in bold and underline, respectively.Source: Figure 5 · p. 6, Table III · p. 7
- In OPE, the tracker achieved the best CLE (14.5 pixels) and precision (0.83), and its success rate was 13% higher than SAMF, the second-best tracker.In TRE, the competing trackers’ average performance was higher than in OPE, while this tracker’s average precision and success rate were lower.Source: B. Comparison with the… · p. 6
C. Qualitative evaluation
Across eight representative sequences, the proposed tracker performs better than seven competitors on scale variation, rotation, and cloud interference. Its qualitative results also cover skydiving and ground-based motorcycle footage.
C. Qualitative evaluation
- On three clear-sky sequences, the proposed tracker localizes scale- and rotation-varying aircraft more accurately than STC, SAMF, DSST, and CCT, whose hardcoded scaling fails.The sequences include low-resolution jets and, in airplane 006, background appearance similar to the target.Source: C. Qualitative evaluation · pp. 7–8, Figure 6 · p. 8
- In cloud-interference sequences airplane 016 and big 2, only the proposed approach gives optimal tracking boxes, although most competitors track the aircraft.The interference is more severe in big 2, where the jet flies at lower altitude.Source: C. Qualitative evaluation · pp. 7–8, Figure 6 · p. 8
- For skydiving footage, the proposed tracker handles large, fast scale variations that defeat competitors facing substantial appearance changes from rotation.Clouds and terrain also interfere with tracking in this sequence.Source: C. Qualitative evaluation · pp. 7–8, Figure 6 · p. 8
- On motorcycle footage, competitors track only the motorcycle head, while the proposed tracker gives a minimum bounding box for the object of interest.The comparison is shown for frames #110–#145.Source: C. Qualitative evaluation · pp. 7–8, Figure 6 · p. 8
- The proposed tracker outperforms seven competitors on large scale variation, rotations, and cloud interference across the qualitative evaluation.Figure 6 shows representative frames from eight sequences.Source: C. Qualitative evaluation · pp. 7–8, Figure 6 · p. 8
D. Limitation
Occlusion can cause tracking failure and degrade performance, although the tracker can recover after several frames or re-localize the object when it reappears. The conclusion also reports automatic initialization, fastest speed, and better performance than competing trackers.
D. Limitation
- Smoke and cloud occlusion can cause tracking failure and deteriorate performance, although tracking recovers after a few frames.In airplane 005, the aircraft is partially occluded at frame #86 and severely occluded at frame #96; recovery is shown at frame #102.Source: D. Limitation · p. 8, Figure 7 · p. 8
V. CONCLUSION
- The conclusion reports automatic initialization, the fastest speed, and better performance than competing state-of-the-art trackers.Source: §V · p. 8
- The tracker cannot handle occluded scenes well, but can automatically re-localize and track the salient object when it reappears in view.Source: §V · p. 8