Source-linked AI summary
F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds
Qi Chen
TL;DR
Autonomous vehicles need more precise real-time perception despite limited sensor views and constrained network bandwidth. F-Cooper uses voxel and spatial feature-level fusion for connected vehicles and edge systems; experiments report better detection, lightweight deployment, and acceptable data volume and transmission time. The paper also identifies sensitivity to low-resolution spatial features for far and small objects and limits its latency simulation to DSRC.
Problem
Autonomous-vehicle perception is constrained by limited sensor views, while sharing raw data is challenged by network bandwidth and real-time requirements.
Method
F-Cooper performs end-to-end 3D object detection by fusing voxel and spatial CNN features on in-vehicle or roadside edge systems.
Results
F-Cooper’s voxel and spatial fusion strategies perform better than the single-vehicle baseline, while feature data size and transmission time remain suitable for edge deployment.
Takeaways & Limitations
Feature fusion provides detection enhancement with lightweight computation and supports edge deployment under limited-bandwidth conditions.
Takeaways & Limitations
Spatial feature fusion has lower feature-map resolution that particularly affects far and small-object detection, and latency experiments simulate only DSRC channels.
Abstract
from arXiv · showhide
Autonomous vehicles are heavily reliant upon their sensors to perfect the perception of surrounding environments, however, with the current state of technology, the data which a vehicle uses is confined to that from its own sensors. Data sharing between vehicles and/or edge servers is limited by the available network bandwidth and the stringent real-time constraints of autonomous driving applications. To address these issues, we propose a point cloud feature based cooperative perception framework (F-Cooper) for connected autonomous vehicles to achieve a better object detection precision. Not only will feature based data be sufficient for the training process, we also use the features' intrinsically small size to achieve real-time edge computing, without running the risk of congesting the network. Our experiment results show that by fusing features, we are able to achieve a better object detection result, around 10% improvement for detection within 20 meters and 30% for further distances, as well as achieve faster edge computing with a low communication delay, requiring 71 milliseconds in certain feature selections. To the best of our knowledge, we are the first to introduce feature-level data fusion to connected autonomous vehicles for the purpose of enhancing object detection and making real-time edge computing on inter-vehicle data feasible for autonomous vehicles.
1 INTRODUCTION
F-Cooper addresses limited vehicle perception and network constraints by fusing CNN-derived point-cloud features for cooperative 3D object detection. The framework targets improved detection while keeping feature transmission and edge deployment lightweight.
- Motivation: CAV perception is challenged by high-volume sensor streams, strict timing requirements, and limited sensor views.Edge computing can combine information from multiple vehicles using more powerful machine-learning technologies.
- Motivation: Cooperative perception can extend an individual vehicle’s perception range by fusing data from nearby vehicles and edge servers.Related work had not fully addressed local accuracy, network impact, exchange format, and edge-side fusion for 3D detection.
- Motivation: Limited sensor capabilities and occlusion make single-source perception risky in real-world environments.Sensors are constrained by scan frequency, range, resolution, and physical obstruction.
- Proposed Solution: F-Cooper performs end-to-end 3D object detection through feature-level fusion, supporting voxel feature fusion and spatial feature fusion.Voxel fusion approaches raw-data-level precision improvement, while spatial fusion allows dynamic adjustment of transmitted feature-map size.
- Efficiency: One hundredth of the original data size enables compressed feature transmission in dozens of milliseconds for real-time edge computing.A typical LiDAR frame is about 4 MB, while compressed CNN features can be as low as 200 Kb.
- Contributions: F-Cooper’s feature fusion improves perception while shifting heavy computation toward edge systems and supporting deployment without infrastructure adjustments.Both voxel and spatial fusion outperform the single-vehicle baseline, with different advantages in precision and compression.
2 TOWARDS FEATURE BASED FUSION OF VEHICLE DATA
The paper motivates feature-based fusion by extracting CNN feature maps that retain object-detection information and location data. Shared formats, coordinate alignment, sparse representation, and lossless compression make these features suitable for vehicle-to-vehicle transmission and edge fusion.
- Feature Maps: CNN convolutional layers transform raw 3D point-cloud data into feature maps used for object detection.Each layer’s filter outputs become inputs to the next convolutional layer.
- Fusion Characteristics: Feature maps may be difficult to fuse losslessly because convolutional-layer specifications can place voxel features at equal distances from one another.The paper identifies this as an issue requiring additional runtime handling.
- Fusion Characteristics: The fusion design assumes compatible vehicles use the same detection model and can combine maps when their point clouds have at least one overlapping point.GPS and IMU data support adjustment for equal-distance location-alignment problems.
- Compression and Transmission: CNN processing filters extraneous raw-data content, and sparse matrices store only information retained as potentially useful for object detection.This representation avoids transmitting unnecessary raw point-cloud data.
- Compression and Transmission: Lossless compression combined with sparse matrices reduces feature data to no more than 1 MB for edge fusion.The paper identifies gzip as an example of lossless compression.
- Feature Maps: Feature maps are considered for fusion because they provide essential detection information while retaining relative location information.The paper states that calibrated LiDAR processing produces maps capable of retaining locations relative to the vehicle.
- Generic and Inherent Properties: Extracted feature maps can be pulled during CNN processing without extra onboard computation or power, using a generic CNN-based network.The paper presents this as avoiding further processing before fusion.
3 F-COOPER: FEATURE BASED COOPERATIVE PERCEPTION
F-Cooper fuses LiDAR-derived features from multiple vehicles using voxel-level or spatial-level schemes before region proposal and object detection. Spatial fusion locally preprocesses features, supports adjustable detection ranges, and reduces transmitted data relative to voxel fusion.
- F-Cooper framework: F-Cooper fuses feature maps from multiple LiDAR sources using Voxel Feature Fusion (VFF) or Spatial Feature Fusion (SFF).VFF fuses VFE-layer voxel features, whereas SFF fuses spatial features produced locally by each vehicle.
- Voxel features: Each non-empty voxel is converted into a fixed-size 128-dimensional feature vector, while empty voxels have no feature.VFE processes voxel point sets; voxels with more than 35 points are randomly sampled to 35 points for efficiency and data balance.
- Voxel features: VFF matches voxel features in calibrated space and applies element-wise maxout to features from coincident or connected voxels.The method handles imperfect alignment by fusing one voxel with all connected voxels when intersections span multiple voxels.
- Fusion operation: Maxout emphasizes prominent features and suppresses trivial ones with negligible additional computational overhead.The operation introduces no extra parameters and is used in both voxel and spatial fusion.
- Spatial feature fusion: SFF reduces communication traffic because spatial feature maps are sparser and more compressible than voxel features, while partial channel transmission can further reduce data.Channels are selected from the 128 available channels based on their differing contributions to detection.
- Spatial feature fusion: SFF first produces spatial feature maps locally, fuses overlapping regions in a larger detection range, and forwards the fused maps to an RPN.Non-overlapping regions retain their original features, while overlapping spatial features are fused with maxout.
4 PERFORMANCE EVALUATION
The evaluation uses KITTI-derived and independently collected LiDAR data across simulated and real vehicle scenarios, including occluded and crowded environments. These datasets support testing feature-based cooperative perception in varied driving conditions.
- Datasets: KITTI supplies labeled 3D Velodyne point clouds, while its single-vehicle recordings are segmented to emulate data from two vehicles.Each frame contains about 100K points with 3D location and reflectance information.
- Datasets: The T&J dataset uses two independently moving vehicles equipped with LiDAR, cameras, radar, IMU/GPS, and edge-computing devices.The setup enables experiments with vehicles that move independently of each other.
- Datasets: Both datasets provide data for 3D object detection and contain sufficient LiDAR information to extract CNN feature maps.
- Test scenarios: Road intersections test perception under vehicle-induced occlusion that severely limits sensor information.
- Test scenarios: Multi-lane roads combine high-speed driving and T-junctions, while campus parking lots represent crowded environments with many obstacles.The parking-lot scenario is the main real-life evaluation setting for fusion-based perception.
4.3 Experiment Setup
F-Cooper is evaluated with more than 200 data sets across four LiDAR-processing categories and scenario-specific object priorities. The experiments compare non-fusion, voxel-feature fusion, spatial-feature fusion, and raw point-cloud fusion, using detection visualizations and confidence thresholds.
- Evaluation design: More than 200 data sets are evaluated using non-fusion, VFF, SFF, and Cooper raw point-cloud fusion.Feature fusion is tested across random cases in the four categories.
- Evaluation design: Objects within 20 meters are treated as high-priority, while objects beyond 20 meters are treated as low-priority in parking-lot experiments.
- Evaluation design: Detection experiments use a 0.5 confidence threshold to mark an object’s bounding box.The same threshold is used for baseline and fusion detections.
- Evaluation design: Yellow boxes indicate receiver-only detections, green boxes indicate sender-only detections, and red boxes indicate detections produced after fusion.
- Voxel-feature fusion: In parallel-driving scenarios, VFF detects vehicles previously occluded from Car 1 or undetected by both cars.Car 1 initially detects four vehicles and Car 2 initially detects three.
- Spatial-feature fusion: In opposing-traffic scenarios, SFF adds two detections for Car 1, whereas raw fusion adds three.The comparison is shown for spatial-feature fusion and raw fused data.
4.5 Detection Precision Analysis
Detection precision is analyzed against baseline and raw fusion using IoU 0.7, with separate near and far ranges divided at 20 meters. VFF matches raw fusion for near detection, while SFF is more competitive near than far.
- Analysis method: Precision is computed at IoU threshold 0.7 for vehicles by comparing detections with ground truth.
- Baseline: Baseline precision is substantially higher for near than far detections across road and parking-lot scenarios.The near/far cutoff is 20 meters from the vehicle.
- Method comparison: VFF achieves precision similar to Cooper for near objects in road scenarios without collecting other vehicles’ point clouds.
- Method comparison: SFF generally exceeds baseline precision but does not outperform VFF, with a pronounced gap between near and far performance.Spatial features are considerably sparser than voxel features and raw data, favoring denser near-range regions.
- Precision improvement: 40% detection precision improvement occurs for nearly 85% of far-category cases with VFF.VFF also achieves 60% improvement for 30% of far-category cases.
- Precision improvement: In near-category cases, both SFF and VFF have a 50% chance of increasing precision by 20%, while SFF reaches 30% improvement at a 30% chance.
- Interpretation: Feature fusion extends detection range and improves precision because points from different vehicles make object representations less sparse.The additional features can also increase detection confidence when they target the same object multiple times.
4.6 Sensitivity and Resilience
The sensitivity analysis examines positional alignment because feature fusion depends heavily on location information. GPS drift has limited effects on missed detections but affects SFF confidence scores more than VFF scores.
- Sensitivity design: Feature-fusion alignment strongly affects final detection precision because fusion relies on location information.The study evaluates both missed detections and confidence-level changes.
- Sensitivity design: The GPS/IMU integration yields less than 10 cm of positional error, while the experiment simulates 0-meter and 0.1-meter GPS drift.
- Detection accuracy: Location drift does not significantly affect SFF detection accuracy when missed detections are considered.
- Confidence sensitivity: VFF confidence changes from GPS drift are slight because increases and decreases across targets approximately balance out.
- Confidence sensitivity: SFF is more sensitive to GPS drifting than VFF based on the average change in confidence scores.
- Limitations: SFF performs worse than VFF for far objects because its lower-resolution feature maps amplify distance and small-object extraction problems.The authors identify surgical voxel-feature extraction as a possible way to address this issue.
4.7 F-Cooper On-Edge
F-Cooper reduces the communication and computation burden of cooperative perception by transmitting selected, compressed features instead of raw point clouds. Voxel and spatial feature fusion support different precision, compression, and latency trade-offs for edge deployment.
- Raw LiDAR transmission is constrained by limited wireless bandwidth, motivating feature-based fusion to avoid network congestion and onboard computation overhead.
- VFF improves perception close to raw-data fusion, while SFF provides better near-range detection than the baseline.
- Less than 1 MB is the final compression size achieved by both feature fusion methods, within DSRC limits.
- 1 Hz is the maximum vehicle-to-vehicle fusion frequency imposed by DSRC, with one frame generally providing the needed supplemental information.
- 55-99 channel SFF achieves the highest compression across five cases, averaging 250 KB, while maintaining similar precision.
- Less than a tenth of a second is required by the fastest strategies to send and receive results from an edge device.
4.8 Summary of Experimental Analysis
F-Cooper's voxel and spatial feature fusion strategies generally improve single-car detection while remaining lightweight for on-edge deployment. The framework trades among latency, prediction compensation, data size, and network bandwidth, but does not fully solve real-time challenges.
- Both VFF and SFF outperform single-car detection results in almost all tests while supporting on-edge deployment without infrastructure changes.
- F-Cooper balances latency, prediction compensation, data size, and network bandwidth, helping detect distant objects before the vehicle reaches them.
- F-Cooper does not fully solve real-time challenges, and the experiments simulate latency only on DSRC channels.
5 RELATED WORK
Prior cooperative-perception research includes low-, feature-, and high-level fusion, but comparatively little work addresses multi-vehicle data fusion at the edge. F-Cooper instead fuses intermediate LiDAR features from multiple vehicles for lightweight on-edge deployment.
- Data fusion methods are commonly grouped into low-level, feature-level, and high-level categories.
- High-level fusion combines detection results from individual vehicles, while prior work also investigates fused results for multi-sensor object detection and tracking.
- Only a few studies had explored fusing data from multiple sources on-edge, including shared situational awareness and collaborative learning systems.
- F-Cooper fuses processed LiDAR features from multiple vehicles, reducing transmitted data for lightweight on-edge deployment and detecting more possible objects than high-level fusion.
6 DISCUSSIONS
The discussion identifies failure modes and scope boundaries for cooperative perception, including missed objects under high-level fusion, localization requirements, and unresolved trust and security concerns.
- High-level fusion can omit an object when separate vehicles each observe only part of it and neither produces a complete detection.
- Perspective transformation requires estimating sender–receiver relative position so transmitted data can be mapped into the receiver's coordinate system.
- AVR uses an offline sparse 3D map as a benchmark for accurate relative vehicle localization and improved fusion precision.
- Security and trust are outside the paper's scope; experiments assume all sources are valid and trustworthy, while authentication is proposed as a possible remedy.
7 CONCLUSIONS
F-Cooper combines voxel and spatial feature fusion for lightweight, computationally inexpensive 3D detection on autonomous vehicle edge systems. Both strategies enhance fusion detection, while spatial features offer greater compression flexibility and feature-only data better suits bandwidth-limited networks.
- F-Cooper performs at the same level as Cooper while being more lightweight and computationally inexpensive.
- Voxel feature fusion outperforms spatial feature fusion, while spatial feature fusion can be adjusted for compression and data transfer.
- Both voxel and spatial feature fusion achieve high detection-perception enhancement over the baseline and remain viable fusion strategies.
- Feature-only 3D LiDAR data is much more suited than raw vehicle-generated data to networks with limited bandwidth.
- F-Cooper's data volume and transmission time fall within an acceptable range for on-edge computation and communication in real-world scenarios.