PaperlayerView exact revision on arXiv

Source-linked AI summary

CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridization

Junhao Xu, Yanan Zhang, Zhi Cai, Di Huang

arXiv:2503.03430v1cs.CV

TL;DR

Collaborative perception must balance communication efficiency with detection accuracy, especially under constrained bandwidth. CoSDH selects regions using supply-demand relationships and combines intermediate fusion with confidence-aware late fusion; it reports state-of-the-art detection accuracy and favorable bandwidth trade-offs across datasets.

  • Problem

    Existing communication-efficient methods can select collaboration regions imprecisely and lose detection accuracy under real bandwidth constraints.

    Source: §1 · p. 1

  • Method

    CoSDH selects sparse collaboration regions using supply-demand relationships and adds confidence-aware late fusion to intermediate feature fusion.

    Source: §1 · pp. 1–2

  • Results

    Across multiple datasets, CoSDH reports state-of-the-art detection accuracy and optimal bandwidth trade-offs, with less degradation and better performance using less bandwidth at a 27 Mbps total limit.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported results support CoSDH’s feasibility for collaborative perception scenarios with communication bandwidth close to real-world limits.

    Source: §5 · p. 8

Abstract

from arXiv · show

Multi-agent collaborative perception enhances perceptual capabilities by utilizing information from multiple agents and is considered a fundamental solution to the problem of weak single-vehicle perception in autonomous driving. However, existing collaborative perception methods face a dilemma between communication efficiency and perception accuracy. To address this issue, we propose a novel communication-efficient collaborative perception framework based on supply-demand awareness and intermediate-late hybridization, dubbed as \mymethodname. By modeling the supply-demand relationship between agents, the framework refines the selection of collaboration regions, reducing unnecessary communication cost while maintaining accuracy. In addition, we innovatively introduce the intermediate-late hybrid collaboration mode, where late-stage collaboration compensates for the performance degradation in collaborative perception under low communication bandwidth. Extensive experiments on multiple datasets, including both simulated and real-world scenarios, demonstrate that \mymethodname~ achieves state-of-the-art detection accuracy and optimal bandwidth trade-offs, delivering superior detection precision under real communication bandwidths, thus proving its effectiveness and practical applicability. The code will be released at https://github.com/Xu2729/CoSDH.

1. Introduction

The introduction frames communication limits as a barrier to preserving collaborative-perception accuracy and proposes CoSDH to refine collaboration regions and compensate intermediate fusion with late fusion. It reports favorable accuracy-bandwidth trade-offs across simulated and real-world datasets.

1. Introduction

  • Where2comm loses about 1.5%, 7.5%, and 14.5% AP@0.7 at 27, 5, and 1 Mbps on OPV2V, respectively, with low-bandwidth accuracy falling below late fusion.The 27 Mbps communication rate is identified as approximately the rate provided by common V2X DSRC.Source: §1 · p. 1
  • Supply-demand modeling distinguishes well-observed low-confidence background from occluded or sparse regions, skipping Area 1 while retaining collaboration for Areas 2–3.The three areas appear similarly low-confidence from Ego’s perspective, but Area 1 is well observed whereas Areas 2 and 3 have poorer observations.Source: §1 · pp. 1–2
  • Across OPV2V, V2XSim, and DAIR-V2X, CoSDH achieves state-of-the-art detection accuracy and optimal bandwidth trade-offs; at 27 Mbps it degrades less and outperforms other methods while using less bandwidth.The 27 Mbps result is reported for a simulated real-world communication condition with a total bandwidth limit.Source: §1 · p. 2
  • CoSDH combines intermediate feature transmission and fusion with confidence-aware late fusion, which compensates for intermediate results at low bandwidth with minimal communication cost.The introduction presents late fusion as a way to improve the accuracy lower bound under low-bandwidth conditions.Source: §1 · p. 2

2. Related Work

The related work describes limits of single-agent detection and trade-offs among collaborative-perception timing strategies. It then reviews communication-saving approaches and motivates combining intermediate and late collaboration.

2.1. 3D Object Detection

  • Single-agent 3D detection methods are limited by sensor range and occlusion, motivating collaborative detection that supplements their observations with information from other agents.The subsection also groups 3D detection methods into image-based, point-cloud-based, and multimodal-fusion-based approaches.Source: §2.1 · pp. 2–3

2.2. Collaborative Perception

  • Early collaboration shares raw data with good accuracy but high bandwidth; late collaboration shares results at lower bandwidth but reduced accuracy, while intermediate collaboration adjusts transmitted features for a trade-off.The subsection notes that many intermediate methods transmit complete, uncompressed BEV features, creating large bandwidth requirements.Source: §2.2 · p. 3
  • Autoencoder compression can degrade accuracy at high compression, and Where2comm accuracy can fall rapidly as bandwidth shrinks; this paper proposes late collaboration to compensate for intermediate results.The paper also adopts autoencoders and information selection to save bandwidth while maintaining accuracy.Source: §2.2 · p. 3

3. Method

CoSDH formulates collaborative 3D detection under per-agent communication budgets and combines supply-demand region selection, multi-scale feature compression and fusion, and confidence-aware late fusion. These components target efficient communication while maintaining detection accuracy.

3.1. Problem Definition

  • The objective is to maximize collaborative 3D object-detection accuracy while keeping each agent’s transmitted message volume within budget B.The model aggregates an agent’s own observations and messages from other agents for 3D object detection.Source: §3.1 · p. 3

3.2. Overall Architecture

  • Each agent builds multi-scale BEV features and demand masks; collaborators select and compress sparse features, which Ego fuses before detection and confidence-aware late fusion.Collaborators send nonzero feature parts and their coordinates; late fusion yields the final hybrid detection results.Source: §3.2 · pp. 3–4

3.3. Supply-Demand-Aware Information Selection

  • CoSDH distinguishes poorly observed low-confidence regions from observable background: demand uses low point-cloud density, while supply selects foreground-confidence regions.Observable background does not require collaboration; distant or occluded regions with low point-cloud density do.Source: §3.3 · p. 4

3.4. Message Compression and Fusion

  • Separate autoencoders compress each feature scale before layer-wise fusion, aiming for more thorough fusion with less bandwidth; max fusion combines features efficiently.The paper contrasts this with single-scale fusion and CoAlign’s compression of large-scale single-layer features.Source: §3.4 · pp. 4–5

3.5. Confidence-Aware Late Fusion

  • Confidence-aware late fusion filters collaborator boxes below ε_l and downweights remaining scores by β∈(0,1), addressing low-confidence false positives and overrides of Ego detections.Naive late collaboration can introduce low-confidence false positives or let suboptimal collaborator results override Ego’s detections.Source: §3.5 · p. 5

4. Experiments

Experiments evaluate CoSDH for LiDAR-based 3D detection across simulated and real-world collaborative-perception datasets under bandwidth constraints. Results show strong accuracy-bandwidth trade-offs, including under simulated real-world communication limits.

4.1. Datasets and Experimental Settings

  • The evaluation covers LiDAR-based 3D detection on OPV2V, V2XSim, and DAIR-V2X, spanning simulated and real-world scenarios and V2V and V2I collaboration.Bandwidth is calculated from average data transmitted per collaborative agent to the Ego agent, assuming 10 Hz detection and a 27 Mbps vehicle communication rate, or 6.75 Mbps per agent for up to four collaborators.Source: §4.1 · p. 6

4.2. Quantitative Evaluation

  • CoSDH achieves the highest accuracy on OPV2V and V2XSim while using about 1/100 to 1/1000 of other non-communication-efficient methods’ bandwidth; on DAIR-V2X, its AP@0.7 exceeds CoAlign’s at about 1/300 the bandwidth.Under simulated real-world rate limits, AP@0.7 improves over previous best methods by 3.06%/2.34%/3.75% on OPV2V/V2XSim/DAIR-V2X, respectively.Source: §4.2 · pp. 6–7, Figure 5 · p. 7
  • With bandwidth reduced by 80% to 85%, CoSDH’s AP decreases by less than 0.3% on OPV2V and DAIR-V2X, with less degradation than Where2comm.Figure 5 compares accuracy-bandwidth trade-offs across OPV2V, V2XSim, and DAIR-V2X; CoSDH achieves the best trade-off, with a 27 Mbps total real-world limit and 6.75 Mbps per agent.Source: §4.2 · p. 7, Figure 5 · p. 7

4.3. Qualitative Evaluation

  • On DAIR-V2X under simulated real-world rate limits, CoSDH matches CoAlign and Where2comm’s recall with fewer false positives and more accurate localization.The visual comparison also shows late fusion compensating for intermediate fusion under low bandwidth and improving object recall.Source: §4.3 · p. 8, Figure 6 · p. 8

4.4. Ablation Studies

  • On OPV2V, supply-mask selection reduces bandwidth by about 95% with a slight accuracy decrease, and adding the demand mask cuts bandwidth by about 10% without significant accuracy loss.Autoencoder compression reduces bandwidth by its compression factor without significant accuracy loss; FP16 nearly halves bandwidth without significant loss. Late fusion improves accuracy at a relatively small bandwidth cost, but the improvement is not significant.Source: §4.4 · p. 8, Table 2 · p. 8
  • At 0.05 Mbps of intermediate-fusion bandwidth, late collaboration costs 0.14 Mbps and improves AP@0.5 by 3% and AP@0.7 by 4%.Late collaboration provides more accuracy improvement under lower bandwidth conditions.Source: §4.4 · p. 8, Table 3 · p. 8

5. Conclusion

CoSDH combines supply-demand-aware region selection with confidence-aware late fusion to improve communication efficiency and accuracy under bandwidth constraints.

5. Conclusion

  • CoSDH models supply-demand relationships to select sparse, key collaboration regions and adds confidence-aware late fusion to intermediate collaboration for 3D object detection.Experiments across multiple datasets report a better accuracy-bandwidth trade-off and strong accuracy under bandwidth constraints close to real-world communication limits.Source: §5 · p. 8

A. Dataset Details

The appendix describes three collaborative-perception datasets used for 3D detection, including two simulated datasets and a real-world vehicle-to-infrastructure dataset.

A. Dataset Details

  • OPV2V is a large-scale V2V collaborative-perception simulation dataset built using OpenCDA and CARLA.It contains 11,464 frames, with 6,374 for training, 2,170 for validation, and 1,980 for testing.Source: A. Dataset Details · p. 12
  • V2XSim is a large-scale V2X collaborative-perception dataset created using SUMO and CARLA.It contains 10,000 frames with annotations for object detection, tracking, and semantic segmentation.Source: A. Dataset Details · p. 12
  • DAIR-V2X is a real-world V2I collaborative-perception dataset collected at an autonomous-driving demonstration intersection in Beijing.Each of its 38,845 frames contains data from a vehicle and roadside infrastructure.Source: A. Dataset Details · p. 12

B. More Visualization

Additional visualizations illustrate how supply-demand masks select regions and how confidence-aware late fusion affects detection results on DAIR-V2X.

B. More Visualization

  • The demand mask covers occluded areas near Ego and sparse-point-cloud regions farther away, while combined supply-demand masks avoid areas Ego observes well.Combining supply masks with the demand mask reduces bandwidth by about 10% while maintaining detection accuracy.Source: B. More Visualization · p. 12
  • On DAIR-V2X, naive late fusion adds suboptimal detections that can lower precision and recall, whereas confidence-aware late fusion avoids them and increases overall AP.The PR curves compare intermediate fusion, naive late fusion, and confidence-aware late fusion.Source: B. More Visualization · p. 12

C. More Ablation

The ablation compares fusion and compression schemes, finding that multi-scale fusion improves accuracy and multi-scale compression reduces bandwidth.

C. More Ablation

  • Multi-scale fusion significantly improves accuracy over traditional single-scale fusion, while multi-scale compression reduces bandwidth and fusion after compression further improves accuracy.The comparison removes supply-demand selection and transmits the complete BEV feature map to isolate the fusion and compression schemes.Source: C. More Ablation · p. 12

D. Discussion About the Latency

CoSDH's latency depends chiefly on communication volume, which supply-demand selection reduces; its two communication rounds remain more than common single-communication methods. It maintains the highest accuracy under 100 ms latency, while at 200 ms almost all collaborative methods perform worse than “No Fusion”.

D. Discussion About the Latency

  • CoSDH uses two rounds of inter-agent communication, more than common single-communication methods.It transmits demand masks, intermediate features, and detection results; single-car detection results are sent for late fusion so they can travel with intermediate features.Source: D. Discussion About the Latency · pp. 12–13
  • CoSDH reduces communication volume by more than 60% while maintaining the highest accuracy, and demand-mask transmission adds only 3 ms to total latency.The authors attribute the modest added latency to parallel processing; transmitting detection results also does not greatly increase total latency.Source: D. Discussion About the Latency · p. 13
  • At 100 ms latency, CoSDH maintains the highest accuracy; at 200 ms, almost all collaborative methods perform worse than “No Fusion”.The discussion attributes CoSDH's latency robustness to multi-scale fusion and collaborative region selection.Source: D. Discussion About the Latency · p. 13, Figure C · p. 13
Loading 2503.03430v1…