Source-linked AI summary
EVPeriscope: Extended Perception across Aerial and Ground Vehicles with Event-based Propeller Tracking
Dexter Ong, Vijay Kumar, Pratik Chaudhari
TL;DR
Aerial–ground teams need reliable relative localization, but conventional cameras and fiducial markers are vulnerable to motion blur, lighting changes, payload limits, and small landing zones. EVPeriscope uses an upward-facing ground-robot event camera to track quadrotor propellers for localization and control, enabling extended aerial perception when ground sensors are occluded. Field experiments show operation in dense foliage, at night, and under wind disturbances, with 200 Hz onboard control.
Problem
Reliable relative localization is needed for coordinated aerial–ground robots, but conventional cameras and fiducial markers face motion blur, lighting, payload, and landing-zone constraints.
Method
EVPeriscope uses an upward-facing event camera on a UGV to detect and track quadrotor propellers for markerless pose estimation, control, and extended perception.
Results
The system supports localization and navigation through dense foliage, nighttime and wind-disturbed operation, and fully onboard quadrotor control at 200 Hz.
Takeaways & Limitations
The UAV can extend the UGV’s sensing and mapping coverage when ground-level sensors are occluded in challenging outdoor environments.
Takeaways & Limitations
Performance when UGV sensors are occluded is limited by the UAV’s localization sensor and algorithm, and the system cannot handle high-frequency propeller noise from another UAV.
Abstract
from arXiv · showhide
Reliable relative localization between aerial and ground robots is a key requirement for tightly coordinated heterogeneous teams. This can be difficult to do using conventional frame-based cameras and fiducial markers because they are sensitive to motion blur, lighting variations, and payload constraints. This paper presents EVPeriscope, an event-based perception system that enables detection, localization and control of a quadrotor using an upward-facing event camera on a ground robot by detecting the high-frequency visual signature of its propellers. This system allows the quadrotor to function as an extended perception system for the ground robot when onboard sensors exhibit degradation or occlusion. We demonstrate the capabilities of this marsupial ground-aerial system via experiments in challenging field conditions with wind speeds of up to 15 mph, in both daylight and at night. We show that the system supports localization and closed-loop navigation through dense foliage where the ground robot's sensors are occluded. Our control system for the quadrotor operates at 200 Hz entirely with onboard sensing and computation. More details and experiment videos can be found on the project page: https://ongdexter.github.io/evperiscope.
1 Introduction
EVPeriscope addresses marker and camera limitations in aerial–ground coordination with a markerless event-based system that detects quadrotor propellers from a ground robot. The system enables 200 Hz pose estimation and control, precision landing, and extended aerial perception when ground sensing is degraded or occluded.
- Motivation: Marker-based and frame-camera approaches face payload, airflow, motion-blur, lighting, and landing-zone size constraints that motivate markerless sensing.Fiducials may require added UAV or UGV hardware and sufficient physical area for reliable detection.
- System capabilities: The tightly coupled marsupial system supports precision landing on the UGV and uses the UAV’s elevated viewpoint to augment sensing during mapping and navigation.This architecture combines UGV endurance and computation with UAV viewpoint diversity.
- Event-based sensing: Event cameras capture rapid brightness changes with microsecond temporal resolution and high dynamic range, preserving propeller visibility under difficult lighting.Known propeller geometry is used to recover quadrotor pose for relative localization and control.
- System overview: EVPeriscope combines an upward-facing UGV event camera with propeller tracking for markerless quadrotor localization and control at 200 Hz.The system uses the UAV as an extended perception platform for the ground robot.
- Validation: Field validation covers dense foliage, nighttime conditions, and wind disturbances using fully onboard sensing and computation.These experiments demonstrate the system’s operation under challenging outdoor conditions.
2 Related Work
Prior work demonstrates event-based UAV detection for challenging lighting, low-power operation, and surveillance, while other localization systems address GPS or ultra-wideband positioning. EVPeriscope instead uses a lightweight event-driven representation for persistent full-pose relative localization and tightly coupled aerial–ground coordination.
- Event-based UAV detection: Earlier event-based approaches use synthetic propeller streams, convolutional detectors, spiking networks, or dynamics-focused pipelines for UAV detection and tracking.These methods address detection and tracking across different operating and power constraints.
- Event-based UAV detection: EVPeriscope differs from learning-based event detectors by operating directly on incoming events without a trained model or propeller-specific training data.The system targets persistent relative localization of the UAV’s full pose for heterogeneous coordination.
- Relative localization: GPS can have meter-scale noise without RTK correction, while ultra-wideband systems provide accuracy on the order of tens of centimeters that may remain insufficient for small landing zones.These limitations motivate more accurate relative localization for precision landing.
- Marsupial systems: Marsupial robot teams commonly deploy UAVs for complementary exploration or mapping, but typically do not account for recovery and repeated deployments.EVPeriscope is positioned within this carrier–child robot paradigm.
3 Method
EVPeriscope localizes and controls a quadrotor from an upward-facing UGV event camera by tracking propeller activity and exploiting known vehicle geometry. The pipeline integrates event processing, pose estimation, control, and hardware-software communication for closed-loop operation.
- System Overview: The UGV event camera detects and tracks propellers, estimates UAV pose from their asymmetric geometry, and uses that pose to command UAV velocities.UAV observations and state estimates are also transmitted to the UGV as extended perception.
- 3.1 Event-based Propeller Detection and Tracking: A persistent event image applies leaky temporal integration, emphasizing recent events while suppressing older and noisy activity.Events are processed in 5 ms windows with a 20 ms decay time constant fixed across experiments.
- 3.1 Event-based Propeller Detection and Tracking: The detector thresholds and morphologically cleans the event image, then uses k-means with k = 4 to obtain one centroid for each propeller.Centroids from the previous image initialize k-means, and the approach can in principle extend to N rotors by setting k = N.
- 3.2 Quadrotor Pose Estimation: Known asymmetric quadrotor geometry establishes centroid correspondences, enabling PnP initialization and EKF updates of the UAV’s full 6-DoF pose.The method requires visibility of all four propeller centroids in this work; three would be sufficient in principle.
- 3.3 UAV Control and 3.4 UAV Takeoff and Landing: The UGV controller tracks position and yaw errors in its camera frame and sends velocity commands after stable UAV localization.Takeoff initializes the UAV at 1.5 m, while landing uses a 0.5 m height and requires staying within 0.05 m of the target for 3 s.
4 Experiments
EVPeriscope was evaluated for UAV pose estimation, nighttime tracking, localization under foliage occlusion, elevated mapping, and closed-loop navigation. Across these tests, the event-based system maintained useful perception and enabled real-time cooperative autonomy in challenging conditions.
- 4.1 UAV Detection and Pose Estimation: 0.03 m positional error near the camera and less than 0.7 m at 6 m height demonstrate localization across the tested altitude range.Pose error increases with distance because of the event camera’s resolution and field of view.
- 4.1 UAV Detection and Pose Estimation: Conventional marker detection was inconsistent at 0.5 m and failed beyond that, whereas EVPeriscope consistently detected propellers across lighting conditions.The comparison highlights event-based sensing under harsh outdoor illumination.
- 4.1 UAV Detection and Pose Estimation: LED propellers enabled propeller detection and UAV tracking in the dark, extending operation to extreme low-light conditions.The nighttime experiment used GPS for UAV state estimation because visual-inertial odometry was severely degraded.
- 4.2 Localization under Sensor Occlusion: EVPeriscope maintained good perception and localization above dense foliage while forward-facing UGV VIO failed and LiDAR retained noisy estimates.The elevated UAV viewpoint remained unobstructed when ground sensors were occluded.
- 4.3 Mapping with Extended Perception: EVPeriscope mapped more environment than the UGV camera by using the UAV’s elevated viewpoint to reach areas beyond the ground sensors’ vertical field of view.The result is relevant to higher-elevation targets such as trees and buildings.
5 Discussion and Limitations
The discussion identifies limits tied to UAV localization quality and interference from other UAV propellers, while describing tighter fusion with UGV state estimation when ground sensors remain available.
- 5 Discussion and Limitations: When UGV sensors are occluded, EVPeriscope state-estimation quality is limited by the UAV’s localization sensor and algorithm, here visual-inertial odometry.A larger UAV with LiDAR could improve localization, but UGV payload constraints remain relevant.
- 5 Discussion and Limitations: The current system cannot handle high-frequency noise from another UAV’s propellers, motivating future identification and assignment of propellers to individual UAVs.This bounds the demonstrated tracking capability to scenarios without that interference.
- 5 Discussion and Limitations: Relative pose remains reliable despite possible UAV VIO drift when unobstructed UGV sensing is fused with the UGV’s more accurate state estimate.The UGV’s LiDAR provides higher-accuracy state estimation in this operating condition.
6 Conclusion
EVPeriscope uses an upward-facing event camera on a ground robot to detect, localize, and control a quadrotor without external infrastructure. Real-world experiments showed reliable propeller detection, navigation through foliage-occluded areas, and expanded mapping coverage using the UAV as an elevated sensor.
- EVPeriscope enables tightly coupled ground–aerial coordination by detecting, localizing, and controlling a quadrotor from an upward-facing ground-robot event camera.
- Real-world experiments showed reliable event-based propeller detection under challenging outdoor lighting, localization and navigation through dense foliage, and expanded mapping coverage from an elevated UAV platform.