PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse2comm: Towards Robust Cooperative 3D Object Detection

Lei Yang, Boqi Li, Chunmian Lin, Li Wang, Ziying Song, Shaoqing Xu, Heye Huang, Haibao Yu, Chen Lv

arXiv:2610.08573v1cs.CV

TL;DR

Limited bandwidth and unreliable cooperation leave coupled packet loss, delay, and spatial misalignment insufficiently addressed by methods focused on communication cost or individual degradations. Sparse2comm progressively restores cooperative features, and across three datasets it improves mixed-setting AP over Where2comm while using 1.0% relative feature bandwidth.

  • Problem

    Bandwidth limits and co-occurring packet loss, delay, and spatial misalignment degrade cooperative features, while existing methods often address communication reduction or one degradation type.

    Source: §1 · pp. 1–2

  • Method

    Sparse2comm progressively restores features: SFE reconstructs dense semantics from sparse foreground messages, LAA aligns delayed features, and SCF estimates residual spatial offsets before fusion.

    Source: §1 · p. 2

  • Results

    +20.15/+11.79, +12.66/+11.07, and +15.36/+12.61 AP@0.5/AP@0.7 over Where2comm on DAIR-V2X, OpenV2V, and V2V4Real, respectively, with 1.0% relative feature bandwidth.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Sparse2comm maintains competitive clean accuracy while reporting a strong trade-off among detection accuracy, robustness, and communication efficiency.

    Source: §1 · p. 2

Abstract

from arXiv · show

Cooperative perception improves autonomous driving by sharing complementary observations among vehicles and roadside infrastructure for 3D object detection. However, practical deployment is constrained by limited bandwidth and unreliable cooperation, where packet loss, transmission delay, and spatial misalignment jointly degrade the cooperative feature stream. Existing methods often reduce communication cost or compensate for one degradation type, leaving coupled disturbances insufficiently addressed. To address this problem, we propose Sparse2comm, a bandwidth-efficient and robust cooperative 3D object detection framework that treats unreliable cooperation as progressive restoration over degraded cooperative features. Sparse Feature Encoding first encodes communication as randomly mask-sampled foreground features transmitted by collaborating agents, from which the ego vehicle reconstructs dense semantic representations. This sparse-to-dense mechanism learns to infer missing object-centric content from sparse observations, enabling ultra-low-bandwidth communication and packet-loss recovery within the same representation. On the semantically restored features, Latency-Aware Alignment predicts motion flow to compensate delayed messages, and Self-Calibrating Fusion estimates residual spatial offsets in a self-supervised manner before adaptive cross-agent fusion. Sparse2comm therefore restores semantic completeness, temporal consistency, and spatial alignment in an ordered pipeline. Extensive experiments on DAIR-V2X, OpenV2V, and V2V4Real show that Sparse2comm maintains competitive clean accuracy and consistently improves robustness under individual and mixed real-world degradations. Compared with the selective feature communication baseline Where2comm, Sparse2comm improves mixed-setting AP@0.5/AP@0.7 by +20.15/+11.79, +12.66/+11.07, and +15.36/+12.61 on the three datasets, respectively.

1 INTRODUCTION

Sparse2comm treats bandwidth limits, packet loss, delay, and spatial misalignment as coupled degradations, restoring cooperative features in sequence. It maintains competitive clean accuracy and improves mixed-setting detection over Where2comm across three datasets.

Motivation

  • Packet loss, transmission delay, and spatial misalignment can co-occur, leaving cooperative messages sparse, incomplete, temporally stale, and spatially biased.Existing methods often address communication reduction or a specific degradation, while coupled disturbances remain insufficiently addressed.Source: §1 · pp. 1–2

Proposed approach

  • Sparse2comm frames unreliable cooperation as progressive restoration of a degraded cooperative feature stream before detection.The target representation is compact, semantically complete, temporally synchronized, and spatially calibrated.Source: §1 · p. 2
  • SFE transmits randomly mask-sampled foreground features and reconstructs dense semantics, using sparse-to-dense inference for both low-bandwidth communication and packet-loss recovery.The ego vehicle infers missing object-centric content from sparse foreground observations.Source: §1 · pp. 2–3
  • After SFE, LAA predicts motion flow to restore temporal consistency, then SCF estimates residual spatial offsets self-supervisedly before adaptive fusion.The stages are ordered so spatial calibration follows temporal synchronization.Source: §1 · pp. 2–3

Results

  • +20.15/+11.79, +12.66/+11.07, and +15.36/+12.61 AP@0.5/AP@0.7 over Where2comm on DAIR-V2X, OpenV2V, and V2V4Real, respectively, with 1.0% relative feature bandwidth.The evaluation also reports competitive clean accuracy and generalization across vehicle–infrastructure, simulated vehicle–vehicle, and real-world vehicle–vehicle scenarios.Source: §1 · pp. 2–3

2 RELATED WORKS

Ego-centric detection is limited by single-platform sensing, motivating cooperative perception but introducing communication and reliability constraints. Related cooperative methods reduce communication cost or address specific degradations, while missing-feature recoverability and coupled disturbances remain concerns.

2.1 Ego-centric 3D Object Detection

  • Single-platform detectors can miss occluded or distant objects and struggle with complex intersections, motivating exchange of complementary observations across agents.Cooperation extends effective perception range and reduces blind spots, while introducing communication and reliability constraints.Source: §2.1 · p. 3

2.2 Cooperative Perception

  • Early fusion preserves rich raw observations but communicates heavily, late fusion is efficient but discards dense spatial evidence, and intermediate fusion balances accuracy and bandwidth.Most methods are evaluated assuming received messages remain usable after transmission.Source: §2.2 · p. 3
  • Selective, pragmatic, and object-aware methods reduce transmission cost, but sparsification and packet loss both remove cooperative feature content.This coupling motivates representations that remain recoverable under missing observations.Source: §2.2 · p. 3
  • Prior work addresses packet loss, delay, or spatial misalignment, but disturbances often occur together and should be evaluated in coupled settings.Robustness studies include packet loss, delay, spatial misalignment, and their combined setting.Source: §2.2 · p. 3

3.1 Problem Definition

The problem is cooperative 3D object detection: an ego vehicle fuses complementary features from collaborating agents under a communication budget. Sparse2comm seeks a compact representation that restores semantic completeness, temporal synchronization, and spatial calibration before detection.

Problem formulation

  • Cooperative detection fuses features from nearby vehicles or roadside units with ego-vehicle features under a communication budget.The model predicts 3D bounding boxes for the ego vehicle from its feature and the transmitted collaborator features.Source: §3.1 · p. 3
  • The aggregate transmitted feature volume is constrained by bandwidth C, and the delay between a collaborator and ego vehicle is τ = tv − ti.The collaborator feature is associated with timestamp ti, while the ego feature is associated with tv.Source: §3.1 · pp. 3–4
  • Received cooperative features may be degraded by packet loss, temporal asynchrony, and spatial misalignment, so the target is a compact, complete, synchronized, calibrated representation.The representation is recovered before final detection.Source: §3.1 · pp. 3–4

3.2 Overall Architecture

Sparse2comm extracts collaborator and ego BEV features, then progressively processes collaborator features before fusing and decoding them for detection. Its architecture filters object-foreground features, transmits sparse masked features, reconstructs them, and applies temporal and spatial alignment.

Collaborator and ego feature extraction

  • The collaborator encoder extracts BEV feature sequences, and an RPN predicts ROI boxes that guide an object-aware filter to remove redundant background responses.The ego encoder uses the same PFE-ResNet backbone to extract ego-vehicle BEV features.Source: §3.2 · p. 4

Sparse Feature Encoding

  • SFE sends masked sparse object-foreground features instead of dense feature maps, which the ego vehicle reconstructs.The sequence begins with collaborator-side feature extraction and object-aware foreground filtering.Source: §3.2 · p. 4

Latency-Aware Alignment

  • LAA uses temporal collaborator features to estimate motion flow and warp recovered features to the ego timestamp.This produces temporally aligned collaborator features for subsequent processing.Source: §3.2 · p. 4

Self-Calibrating Fusion and detection

  • After transformation into ego coordinates, SCF estimates residual spatial offsets and fuses calibrated collaborator features with ego features.The fused feature is decoded by a detection head to predict ego-vehicle 3D boxes.Source: §3.2 · p. 4

3.3 Sparse Feature Encoding

Sparse Feature Encoding treats bandwidth reduction and packet loss as a shared missing-content reconstruction problem: collaborators transmit sparse object-foreground features, and the ego vehicle reconstructs dense semantics before later alignment.

3.3 Sparse Feature Encoding

  • Only about 1% of conventional dense feature-level communication needs to be transmitted, while the same reconstruction mechanism handles intentional omissions and packet loss.The ego vehicle reconstructs dense semantic representations from the sparse observations.Source: §3.3 · p. 4

3.3.1 Mask Sampling

  • Object-wise random sampling retains partial foreground patches per instance, while the missing set includes both intentionally omitted and packet-lost patches.The sampling design reduces the risk of removing an entire object and exposes reconstruction to varied sparsity patterns.Source: §3.3.1 · p. 5

3.3.2 Feature Reconstruction

  • An encoder–decoder learns an object-centric BEV reconstruction prior, restoring incomplete foreground features before temporal or spatial correction.It reconstructs patches missing through intentional mask sampling or packet loss.Source: §3.3.2 · pp. 5–6

3.4 Latency-Aware Alignment

Latency-Aware Alignment addresses stale collaborator features by predicting motion flow from temporal BEV features and warping recovered features to the ego-vehicle timestamp. Future collaborator frames provide flow supervision.

3.4 Latency-Aware Alignment

  • After semantic restoration, Latency-Aware Alignment warps stale collaborator features to the ego-vehicle timestamp to restore temporal consistency.The pipeline places temporal alignment after Sparse Feature Encoding and before spatial calibration.Source: §3.4 · p. 6

3.4.1 Flow Generator

  • The Flow Generator concatenates temporal collaborator BEV features and uses an encoder–decoder to predict pixel-wise motion flow.Its architecture has three downsampling stages, three upsampling stages, and a final convolutional layer.Source: §3.4.1 · p. 6

3.4.2 Temporal Alignment

  • Bilinear warping aligns collaborator features to the ego timestamp; training warps clean features, while inference warps reconstructed features.The flow predictor receives cosine-similarity supervision over future collaborator frames; Δt is the sampling interval and n counts the future frames used.Source: §3.4.2 · p. 6

3.5 Self-Calibrating Fusion

Self-Calibrating Fusion addresses residual spatial error after temporal alignment by estimating local feature offsets, then adaptively fusing the calibrated collaborator and ego features.

3.5 Self-Calibrating Fusion

  • SCF estimates local residual offsets after temporal synchronization rather than correcting the global pose, then refines adaptive cross-agent fusion.Temporal alignment and spatial calibration are separated because both delay-induced motion and pose-induced bias appear as BEV displacement.Source: §3.5 · p. 6

3.5.1 Self-Calibrating Alignment

  • Self-Calibrating Alignment predicts pixel-wise offsets from collaborator and ego features, training on synthetic pose perturbations to recover clean transformed features without manual offset labels.At inference, the deployed-pose collaborator feature is calibrated using the learned offset field.Source: §3.5.1 · p. 7

3.5.2 Adaptive Feature Fusion

  • Adaptive Feature Fusion concatenates calibrated collaborator and ego features, refines them with a residual block, and reweights channels with squeeze-and-excitation attention.The resulting representation combines cross-agent information after semantic restoration, temporal synchronization, and spatial calibration.Source: §3.5.2 · p. 7

3.6 Detection Head and Loss Function

A lightweight, single-stage detection head decodes fused BEV features, while the overall objective combines detection supervision with losses for the three restoration stages.

3.6.1 Detection Head

  • The single-stage detection head uses classification, 3D box regression, and direction-classification branches to decode fused BEV features into 3D boxes.The box parameters are x, y, z, l, w, h, and θ.Source: §3.6.1 · p. 7

3.6.2 Overall Loss Function

  • The training objective combines 3D detection loss with auxiliary semantic reconstruction, motion-flow, and residual spatial-calibration losses.λ1, λ2, and λ3 balance the auxiliary losses.Source: §3.6.2 · p. 7

4 EXPERIMENTS

Across three cooperative benchmarks, Sparse2comm maintains competitive clean performance and leads under mixed disturbances; experiments also assess individual robustness, design choices, and efficiency.

4.1 Datasets / 4.2 Implementation Details

  • DAIR-V2X, OpenV2V, and V2V4Real cover vehicle–infrastructure, simulated V2V, and real V2V settings, evaluated with AP@0.5/0.7 and shared packet-loss, 200-ms delay, and pose-noise protocols.Training uses collaborator encoder and RPN pre-training, Sparse Feature Encoding pre-training, then end-to-end cooperative fine-tuning.Source: §4.1 · pp. 7–8, §4.2 · p. 8

4.3.1 Comparison with State-of-the-art

  • +15.75/+9.37, +7.13/+4.11, and +11.11/+7.34 AP@0.5/AP@0.7 over strongest prior methods under mixed degradation on DAIR-V2X, OpenV2V, and V2V4Real, respectively.Under clean communication, Sparse2comm ranks first on DAIR-V2X, second on OpenV2V, and remains competitive on V2V4Real while using 1.0% relative bandwidth.Source: Table 1 · p. 8, §4.3.1 · pp. 8–9

4.3.2 Robustness to Real-world Challenges

  • On DAIR-V2X, Sparse2comm leads under packet loss and spatial misalignment; under delay it achieves the best AP@0.5 and remains close to the best AP@0.7.It reaches 67.12/50.77 AP@0.5/AP@0.7 under packet loss and 63.08/45.39 under spatial misalignment; Fig. 7 traces performance as disturbances strengthen.Source: Table 2 · p. 9, §4.3.2 · pp. 9–10, Figure 7 · p. 9

4.4.1 Effectiveness of Major Components / 4.4.2 Impact of Training Strategy

  • +20.15 AP@0.5 over the baseline under noisy conditions with SFE, LAA, and SCF combined; staged pre-training and fine-tuning outperforms direct joint training.The component ablation reports the full framework's best performance; the training ablation reports more stable optimization and stronger generalization to real-world perturbations.Source: Table 3 · p. 10, §4.4.1 · p. 10, Table 4 · p. 10

4.4.3 Impact of Sampling Strategy

  • Object-aware Sampling outperforms Uniform and Random Sampling in the feature-masking ablation.Uniform sampling is evenly spaced, while Random Sampling assigns equal probability to all regions.Source: Table 5 · p. 10, §4.4.3 · p. 10

4.4.4 Beyond Data Augmentation

  • Under noise, Sparse2comm reaches 58.10/39.54 AP@0.5/AP@0.7, exceeding the corresponding baseline by +20.15/+11.79; augmentation controls show those gains do not originate from augmentation alone.The controls find that random masking or pose disturbance gives the baseline only marginal improvement under noise.Source: Table 6 · p. 10, §4.4.4 · p. 10

4.4.5 Efficiency Analysis

  • At 2.6 Mbps, Sparse2comm achieves 95.7 ms end-to-end latency under 10 Mbps and 74.9 ms under 50 Mbps; its complete system latency rises from 51.2 ms to 69.7 ms.The efficiency comparison accounts for transmission time as well as model inference latency.Source: Table 7 · p. 11, §4.4.5 · p. 11, Table 8 · p. 11

4.4.6 SFE Architecture

  • The balanced SFE architecture S3 (NE=5, ND=2) obtains the highest accuracy in both ideal and noisy conditions, while shallow, unbalanced, or deeper alternatives provide less balance.S3 reaches 69.42/53.78 under ideal conditions and 58.10/39.54 under noisy conditions.Source: Table 9 · p. 11, §4.4.6 · p. 11

4.5 Qualitative Results

Visualizations show Sparse2comm maintaining more accurate detections under mixed communication noise and reconstructing BEV features close to the refined representation despite severe packet loss.

4.5.1 Detection Result Visualization

  • On DAIR-V2X and V2V4Real, Sparse2comm preserves more accurate and spatially consistent detections under packet loss, delay, and misalignment.The figures contrast ground-truth and predicted boxes; the accompanying discussion reports fewer misses and closer box placement in challenging regions.Source: §4.5.1 · p. 12, Figure 8 · p. 12, Figure 9 · p. 12

4.5.2 SFE Reconstruction Visualization

  • Across packet-loss ratios from 10% to 100%, reconstructed BEV features remain structurally close to refined features, with reconstruction loss near zero.Mask-sampled features show severe degradation and large missing regions, while the reconstruction recovers missing content, including at 90% packet loss.Source: §4.5.2 · pp. 12–13, Figure 10 · p. 13
  • Sparse sampling supports reduced bandwidth while the ego vehicle reconstructs dense features with minimal performance degradation.The visualization presents refined, mask-sampled, and reconstructed BEV features alongside information-loss maps and histograms.Source: §4.5.2 · pp. 12–13

5 CONCLUSION

Sparse2comm treats bandwidth limits, packet loss, delay, and spatial misalignment as coupled degradations and addresses them through progressive feature restoration. Across three datasets, it improves mixed-setting AP over Where2comm while using 1.0% relative feature bandwidth.

5 CONCLUSION

  • Sparse2comm addresses coupled cooperative-feature degradations through sparse feature encoding, latency-aware alignment, and self-calibrating fusion.The stages provide sparse transmission and packet-loss recovery, compensate delayed messages, and refine residual pose-induced offsets before adaptive fusion.Source: §5 · pp. 13–14
  • +20.15/+11.79, +12.66/+11.07, and +15.36/+12.61 AP@0.5/AP@0.7 gains over Where2comm are reported on DAIR-V2X, OpenV2V, and V2V4Real, respectively.The comparison uses only 1.0% relative feature bandwidth; clean accuracy remains competitive.Source: §5 · p. 14
  • The current scope is feature-level communication for cooperative 3D object detection; broader autonomy tasks and behavior under dynamic bandwidth and latency remain future work.Named extensions include occupancy perception, trajectory prediction, and end-to-end cooperative driving.Source: §5 · p. 14
Loading 2610.08573v1…