Communication-Efficient Collaborative Perception via Information Filling with Codebook

1. Introduction

• Existing collaborative-perception methods face a perception–communication trade-off, and intermediate collaboration still incurs substantial costs by transmitting high-dimensional feature maps.Late collaboration exchanges detection outputs but yields marginal perception improvements and is vulnerable to noise.Source: §1 · p. 1
• CodeFilling replaces high-dimensional feature transmission with integer code indices and selects non-redundant messages to fill receivers’ information gaps.The selection strategy assumes each agent’s information demand is upper bounded and aims to prevent information overflow among collaborators.Source: §1 · p. 2
• The system combines message representation and selection to optimize collaborative messages, alongside single-agent detection and message decoding and fusion.The codebook provides a shared standardized representation for homogeneous and heterogeneous agents.Source: §1 · p. 2
• 1,333/1,206× less communication cost than Where2comm on DAIR-V2X/OPV2VH+, respectively, with superior performance.CodeFilling also maintains a superior trade-off in homogeneous and heterogeneous settings.Source: §1 · p. 2
• Experiments cover real-world DAIR-V2X and simulation OPV2VH+ datasets, including homogeneous LiDAR and camera settings and heterogeneous collaboration.The reported results include varying communication bandwidths and both collaboration types.Source: §1 · p. 2

2. Related works

• Earlier collaborative-perception methods use handshake or spatial selection and intermediate feature representations, with source coding and channel compression to improve efficiency.The paper identifies accumulated redundant information and continued transmission of high-dimensional feature vectors as limitations of prior methods.Source: §2 · p. 2
• CodeFilling’s system design pairs information-filling-driven message selection with codebook-based message representation to optimize collaborative messages.The figure caption presents these as the two contributions to message optimization.Source: Figure 2 · p. 3
• Task-adaptive codebooks discard task-irrelevant information, but prior work has largely focused on 2D classification rather than collaborative 3D object detection.The paper identifies preserving extensive features for 3D interpretation and adapting to changing bandwidths as challenges for this setting.Source: §2 · p. 3

3. Problem Formulation

• Agents act as both supporters and receivers, exchanging complementary perceptual information in a decentralized network for 3D object detection.Each agent has its own perceptual task and sensor setup, such as LiDAR or camera.Source: §3 · p. 3
• The objective is to maximize all agents’ detection performance subject to a communication budget.The formulation defines a detection model, evaluation metric, transmitted messages, and a measure of communication cost.Source: §3 · p. 3
• The central message-design challenge is to choose transmitted information that is simultaneously informative and compact.The total communication cost across agents must remain within the specified budget.Source: §3 · p. 3

4.1. Single-agent 3D detection

• Agents accept RGB images or 3D point clouds and project perceptual information into a unified global BEV coordinate system.The observation encoder extracts BEV feature maps; the detection decoder converts features into class and regression outputs.Source: §4.1 · p. 3

4.2. Multi-agent collaboration

• Each agent acts as supporter and receiver, selecting and representing messages before decoding and fusing collaborators’ messages to improve perception.Selection and representation address spatial and channel dimensions, respectively.Source: §4.2 · p. 4

4.2.1 Information-filling-driven message selection

• Supporters disclose spatial information-score maps, then locally select higher-scoring regions until their combined information meets each receiver’s demand.The selection utility rises with accumulated information scores up to demand u; its cutoff prevents redundant overfilling.Source: §4.2.1 · p. 4
• Collective selection reduces redundancy across supporters and adjusts information demand to communication conditions, trading efficiency under limited budgets for performance when budgets are ample.The approach assumes accumulated information scores mirror the benefits of feature aggregation and uses selected-region count as a bandwidth proxy.Source: §4.2.1 · pp. 4–5

4.2.2 Codebook-based message representation

• A shared task-driven codebook approximates selected feature vectors with relevant codes, so agents transmit integer indices instead of full floating-point feature vectors.The codebook is learned from agents’ BEV features across training scenes; its representation is lossy for reconstruction while retaining information needed for detection.Source: §4.2.2 · p. 5

4.2.3 Message decoding and fusion

• Received indices are decoded with the shared codebook, and pointwise maximum fusion aggregates reconstructed collaborator features with the receiving agent’s features for upgraded detections.The maximum is taken across connected collaborators at each spatial location.Source: §4.2.3 · p. 5
• CodeFilling achieves the best perception-communication trade-off on DAIR-V2X and OPV2VH+ in homogeneous and heterogeneous settings.The supplied figure captions report this comparison for each dataset.Source: Figure 5 · p. 6, Figure 6 · p. 6

4.3. Loss functions

• Training supervises information-score-map generation, object detection, and codebook learning, with the information-score generator reusing detection-decoder parameters.Network parameters and the codebook are updated simultaneously.Source: §4.3 · p. 6

5.1. Datasets and experimental settings

• DAIR-V2X provides real-world vehicle–roadside-unit scenes, while OPV2VH+ extends vehicle-to-vehicle simulation to 10 agents; evaluation covers AP at IoU thresholds 0.30 and 0.50.The heterogeneous setting balances LiDAR and camera agents; communication volume is measured in log2 bytes.Source: §5 · p. 6, §5.1 · p. 6

5.2. Quantitative evaluation

• CodeFilling achieves a superior perception-communication trade-off across bandwidths and settings, with communication costs 1333/115/863 and 1206/1078/252 times lower on DAIR-V2X and OPV2VH+.The comparisons include LiDAR-only, camera-only, and heterogeneous 3D detection.Source: Figure 9 · p. 7
• CodeFilling inference takes 36/99 ms on DAIR-V2X/OPV2VH+, comparable to Where2comm at 34/94 ms and faster than HMViT at 90/1266 ms.The authors state that this communication efficiency allows agents to collaborate actively.Source: §5.2 · p. 7
• Across pose-error tests using Gaussian noise with mean 0 m and standard deviations from 0–1.0 m, and latency tests from 0–500 ms, perception generally declines as disturbances rise, while CodeFilling consistently outperforms baselines under all imperfect conditions.Other baselines fail when pose error exceeds 0.6 m and latency surpasses 100 ms.Source: §5.2 · p. 7

5.3. Ablation studies

• Code-index representation reduces communication cost 208 times at unchanged detection performance, while information-filling selection raises detection performance 4.8% at the same cost.Across bandwidth conditions, sum-based utility outperforms max-based utility and no selection; codebook size and quantity trade performance against communication efficiency.Source: §5.3 · p. 7, Figure 11 · p. 8
• Lower information demand is more efficient under limited communication budgets, whereas higher demand performs better with ample budgets.Adjusting demand maintains a superior performance-communication trade-off across communication conditions.Source: §5.3 · p. 7, Figure 11 · p. 8

5.4. Qualitative evaluation

• Visualizations show collaboration recovering detections missed by the ego agent while avoiding redundant overlap; selected messages are spatially sparse and unify heterogeneous inputs.CodeFilling also qualitatively outperforms previous SOTAs with 1256 times less communication cost.Source: §5.4 · p. 8, Figure 10 · p. 8, Figure 11 · p. 8

• CodeFilling combines codebook-based message representation and information-filling-driven message selection in a communication-efficient collaborative 3D detection system.Source: §6 · p. 8
• Experiments across real-world and simulation scenarios report state-of-the-art perception-communication trade-offs across LiDAR, camera, and heterogeneous settings, with robustness to pose error and latency.Source: §6 · p. 8
• Future work will explore the temporal dimension and identify critical timestamps.Source: §6 · p. 8