PaperlayerView exact revision on arXiv

Source-linked AI summary

FFBL-Coop: Association-Decoupled Cooperative 3D Multi-Object Tracking

Haoxin Wu, Xiaokai Bai

arXiv:2610.01750v1cs.CV

TL;DR

A shared cross-agent match can let correspondence errors compromise both evidence integration and identity continuity, even when uncertain identity does not eliminate detection value. FFBL-Coop admits and refines cooperative evidence before binding identities with learned association and persistent mappings. It improves detection and tracking on V2X-Seq and Griffin-25M, with reported AP/AMOTA gains over SparseCoop.

  • Problem

    When one cross-agent match controls query placement and identity inheritance, correspondence errors can compromise both representation updates and trajectory continuity.

    Source: §1 · p. 1

  • Method

    FFBL-Coop separates cooperative evidence admission and refinement from identity assignment, using CSA and URA before CPIA applies learned association and persistent mappings.

    Source: §1 · pp. 1–2

  • Results

    AP/AMOTA reaches 54.8/61.1 on V2X-Seq and 65.3/68.8 on Griffin-25M, exceeding SparseCoop by 1.8/19.0 and 9.4/17.9 points, respectively.

    Source: §4.1 · p. 8

  • Takeaways & Limitations

    The framework integrates complementary observations while supporting tracking continuity across changes in query placement.

    Source: §5 · p. 9

    Reported query payload rates exclude packet headers and codebook synchronization, so deployment traffic also depends on update frequency and protocol overhead.

    Source: H METHOD NOTES AND LIMITATIONS · p. 21

Abstract

from arXiv · show

Cooperative 3D tracking must integrate complementary observations across agents and time while maintaining consistent identities. When evidence integration and identity inheritance share a matching decision, errors arising from cross-view appearance differences and spatial misalignment can compromise both feature fusion and track continuity. We propose FFBL-Coop, a fuse first, bind later framework that separates instance admission from identity management. Confidence-ranked Slot Admission (CSA) allocates cooperative queries to available ego slots using confidence and spatial proximity. Unified Representation Aggregation (URA) uses cooperative semantic features and aligned anchors to guide ego-feature retrieval, refining the augmented query bank within a shared transformer decoder. After refinement, Cooperative-Priority Identity Anchoring (CPIA) combines learned association with persistent mappings to establish accepted identity assignments across frames. A shared codebook reduces transmitted payload while retaining AP and AMOTA close to the uncompressed variant. FFBL-Coop achieves AMOTA/AP of 0.611/0.548 on V2X-Seq and 0.688/0.653 on Griffin-25M. Code will be released.

1 INTRODUCTION

FFBL-Coop separates cooperative evidence integration from identity inheritance, addressing the risk that cross-view correspondence errors can compromise both representation updates and track continuity. Its CSA, URA, and CPIA stages implement this fuse-first, bind-later design, with experiments on V2X-Seq and Griffin-25M.

1 INTRODUCTION

  • Cooperative observations from nearby vehicles, roadside infrastructure, or aerial platforms can reveal objects obscured or outside the ego field of view.The tracking challenge is to integrate this complementary evidence over time while preserving identities across viewpoint and visibility changes.Source: §1 · p. 1
  • FFBL-Coop admits and refines cooperative evidence before assigning ego-track identities, separating integration from the decision about which trajectory an observation continues.This decoupling responds to cross-view appearance differences and alignment errors that can compromise both representation updates and trajectory continuity.Source: §1 · pp. 1–2
  • CSA ranks cooperative queries by confidence and proximity for available slots, URA guides ego-feature retrieval, and CPIA binds identities after refinement using association and persistent mappings.CPIA records accepted assignments and new-track identities in persistent mappings.Source: §1 · pp. 1–2
  • Experiments evaluate detection, tracking, and communication on V2X-Seq and Griffin-25M; codebook compression reduces payload while retaining AP and AMOTA close to uncompressed transmission.The system comparison reports these task measures alongside transmission rates.Source: §1 · p. 2

2 RELATED WORK

Related work spans query-based temporal tracking and cooperative perception. FFBL-Coop is positioned as separating cooperative-query admission from identity inheritance through persistent cross-agent mappings.

2.1 3D MULTI-OBJECT TRACKING

  • Query-based 3D detectors aggregate multi-view image features into sparse object queries, while StreamPETR and Sparse4D incorporate history into object-centric representations (Wang et al., 2022; Liu et al., 2022; Wang et al., 2023; Lin et al., 2023).These temporal-query models provide a foundation for recurrent updates.Source: §2.1 · p. 2
  • MOTR propagates track queries, DQTrack separates object and track queries, and ADA-Track alternates detection and association; FFBL-Coop separates cooperative-query admission from identity inheritance (Zeng et al., 2022).Persistent cross-agent ID mappings support continuity across slot changes.Source: §2.1 · p. 2

2.2 COOPERATIVE PERCEPTION

  • Cooperative perception ranges from feature-level fusion and attention aggregation to learned communication decisions, pose alignment, and sparse object-query exchange (Chen et al., 2019; Xu et al., 2022; Xu et al., 2023; Liu et al., 2020; Hu et al., 2022; Lu et al., 2023; Yu et al., 2025).These approaches address complementary evidence integration, communication selectivity, alignment, or object-level exchange.Source: §2.2 · p. 3
  • CoopTrack, SparseCoop, and Long-SCOPE combine cross-agent association with query aggregation or refinement, while FFBL-Coop separates feature admission from identity inheritance (Zhong et al., 2025; Wang et al., 2026b;c).The cited methods use graph affinities and Hungarian matching, geometric and appearance cues with ego-image context, or contextual descriptors with optimal-transport matching, respectively.Source: §2.2 · p. 3

3 METHOD

FFBL-Coop integrates cooperative queries into the ego bank before assigning identities, using agent-specific perception and alignment to prepare evidence for joint refinement. After decoding, it binds identities and maintains correspondences across frames.

3.1 OVERVIEW

  • Cooperative queries enter the ego bank before track identities are assigned, separating evidence integration from identity commitment.The overview describes codebook reconstruction and coordinate alignment before admission, and a separate output path for valid queries outside ego camera views.Source: §3.1 · pp. 3–4

3.2 AGENT-SPECIFIC PERCEPTION STACKS

  • Roadside units construct BEV features, while ego and aerial queries sample multi-scale image features around projected 3D anchors.Agents expose a shared query interface, and the ego decoder recurrently updates the query bank before jointly refining cooperative and native queries.Source: §3.2 · p. 4

3.3 COMMUNICATION AND CROSS-AGENT ALIGNMENT

  • A shared k-means codebook encodes cooperative instance features by nearest codeword; the ego reconstructs them and aligns states to its frame and timestamp.A learned adapter conditions reconstructed features on relative agent rotation.Source: §3.3 · pp. 4–5

3.4 ASSOCIATION-DECOUPLED COOPERATIVE INTEGRATION

  • CSA ranks cooperative candidates by confidence and allocates them to eligible free ego slots by spatial proximity, without assigning ego-track IDs.Inference processes candidates in descending confidence order and uses each slot at most once.Source: §3.4 · p. 5
  • URA uses cooperative semantics and aligned anchors to retrieve ego-image evidence, then refines admitted and native queries in the shared decoder.A complementary local readout supplies nearby cooperative context to queries outside cooperative-initialized slots; empty neighborhoods skip the update.Source: §3.4 · p. 6
  • CPIA assigns identities after final decoding using learned association and persistent mappings, while preserving decoded features and boxes.Accepted matches inherit track IDs; unresolved hypotheses receive new IDs, and assignments update the next-frame mapping.Source: §3.4 · pp. 6–7

3.5 TRAINING OBJECTIVES

  • Training starts from pretrained single-agent checkpoints, trains cooperative modules and all six decoder layers, and freezes the backbone and FPN.The identity loss is applied once per frame with λid = 1.0.Source: §3.5 · p. 7

4 EXPERIMENTS

Experiments assess FFBL-Coop across V2I and V2D benchmarks, component ablations, camera-projection coverage, calibration perturbations, latency, and communication efficiency. Results show benchmark gains, with detection accuracy more sensitive than aggregate tracking performance in the tested robustness conditions.

4 EXPERIMENTS

  • Experiments span V2I and V2D, using V2X-Seq-SPD at its 2 Hz validation protocol and Griffin-25M for V2D, with car-class AP and AMOTA.AP averages scores at center-distance thresholds of 0.5, 1, 2, and 4 m.Source: §4 · p. 7

4.1 MAIN RESULTS

  • AP/AMOTA reaches 54.8/61.1 on V2X-Seq and 65.3/68.8 on Griffin-25M, exceeding SparseCoop by 1.8/19.0 and 9.4/17.9 points, respectively.Compressed payload is approximately 26.8%/34.6% of SparseCoop's reported rates on V2X-Seq/Griffin-25M, respectively.Source: Table 1 · p. 8, §4.1 · p. 8

4.2 ABLATION STUDY

  • Full improves over Base by 14.9/16.6 AP and 13.4/20.4 AMOTA points on V2X-Seq/Griffin-25M; removing URA produces the largest leave-one-out drops.Removing CPIA lowers AMOTA by 1.8/1.5 points and increases IDS from 70/50 to 78/124; replacing Hungarian allocation with CSA adds 1.0/2.7 AMOTA points. CPIA improves tracking under both allocation rules, although Full does not minimize IDS.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 ROBUSTNESS AND COMMUNICATION EFFICIENCY

  • Both-agent AP rises from 39.2/42.2 to 53.5/72.1 on V2X-Seq/Griffin-25M, while AP in the other projection bins remains nearly unchanged.Shared camera coverage accounts for approximately 22% of evaluated objects on V2X-Seq versus 74% on Griffin-25M; this smaller fraction and smaller within-bin gain help explain the lower overall detection improvement on V2X-Seq.Source: Table 3 · p. 9, §4.2 · p. 9
  • At 1.0° yaw error, AP falls 5.4 points but AMOTA 0.7; at 200 ms delay, AP falls 5.9 points but AMOTA 0.4.Across tested calibration conditions and delays, FFBL-Coop retains higher AP and AMOTA than displayed SparseCoop baselines. With Kcode = 256, payload falls from 1.72 × 105 to 8.48 × 103 B/s with 0.1-point AP and AMOTA drops, while IDS rises from 49 to 70.Source: Table 4 · p. 9, Table 5 · p. 9, Table 6 · p. 9

5 CONCLUSION

FFBL-Coop separates cooperative evidence admission from identity inheritance, then refines admitted evidence before binding identities. On V2X-Seq and Griffin-25M, it improves detection and tracking with compact feature transmission, while perturbation studies show comparatively stable tracking despite reduced detection accuracy.

5 CONCLUSION

  • FFBL-Coop separates cooperative evidence admission from identity inheritance, using CSA, URA, and CPIA to allocate, refine, and bind identities after refinement.The conclusion reports improved detection and tracking on V2X-Seq and Griffin-25M with compact feature transmission; perturbation studies show comparatively stable tracking despite reduced detection accuracy.Source: §5 · p. 9

AI USE STATEMENT

The authors disclose using generative AI tools for writing assistance, retrieval, discovery, ideation, and execution, while taking responsibility for the final content. The supplement outlines additional evaluation, qualitative comparisons, ablations, robustness analyses, and reproducibility protocols.

AI USE STATEMENT

  • Generative AI tools aided or polished writing, retrieval and discovery, and research ideation or execution; the authors verified the work and take responsibility for its final content.The paper says details are described elsewhere in the paper.Source: AI USE STATEMENT · p. 10

A OVERVIEW

  • The supplement covers detailed benchmark outputs, qualitative comparisons, occlusion and range analyses, ablations, robustness and communication efficiency, and reproducibility protocols.Its overview also notes that methodological distinctions and limitations are discussed in Section H.Source: A OVERVIEW · p. 12

B OFFICIAL EVALUATION RECORDS

The official evaluation records report car-class detection and tracking results using codebook-compressed features. They show threshold-dependent AP and identify false negatives as exceeding false positives and identity switches combined at the reported operating points.

B OFFICIAL EVALUATION RECORDS

  • The records cover car-class Full checkpoints with codebook-compressed features, using Kcode = 256 on V2X-Seq and 512 on Griffin-25M.AP, MOTA, and AMOTA are reported in percent.Source: B OFFICIAL EVALUATION RECORDS · p. 12
  • AP rises from 19.95 at 0.5 m to 79.25 at 4 m on V2X-Seq, and from 44.15 to 77.65 on Griffin-25M.The passage describes the gains at relaxed thresholds as indicating that precise localization remains a limitation; ATE and AVE separately characterize center and velocity estimation errors.Source: Table 7 · p. 12, B OFFICIAL EVALUATION RECORDS · p. 12
  • False negatives exceed false positives and identity switches combined on each dataset, making missed objects a substantial error source at the reported operating points.MOTA at selected recall levels complements AMOTA by reporting performance at individual operating points.Source: Table 8 · p. 12, B OFFICIAL EVALUATION RECORDS · p. 12

C ADDITIONAL QUALITATIVE EVIDENCE

Roadside and drone views illustrate detections that are absent or obscured in ego views. Across both datasets, Full improves AP in both occlusion bins over Base.

C ADDITIONAL QUALITATIVE EVIDENCE

D.1 OCCLUSION PROTOCOL

  • Full improves AP over Base in both occlusion bins: by 4.2 and 5.1 points on V2X-Seq, and 20.9 and 38.3 points on Griffin-25M.Objects are grouped as light (occlusion ratio below 0.2) or occluded (at least 0.2).Source: §D.1 · p. 15, Table 9 · p. 15

D.2 RANGE-STRATIFIED RESULTS

Full improves detection across every evaluated range bin, while ablations show that module effects depend on their combination and inference settings.

D.2 RANGE-STRATIFIED RESULTS

  • At 40–50 m, Full improves AP over Base from 33.6 to 40.3 on V2X-Seq and 3.2 to 27.6 on Griffin-25M, while Full@0.5 remains 10.9 and 9.6, respectively.Source: §D.2 · p. 15

E.1 COMPLETE MODULE ABLATIONS

  • Among single-module controls, URA yields the highest AP, CPIA the highest AMOTA on each dataset, and CSA alone improves both metrics over Base.These patterns suggest representation quality and identity handling contribute to aggregate tracking performance.Source: §E.1 · p. 16
  • On Griffin-25M, adding URA to CSA raises AMOTA from 63.4 to 67.3 but IDS from 43 to 124; adding CPIA yields AMOTA 68.8 and IDS 50.Detection effects are non-monotonic: adding CSA to URA-only lowers AP from 64.0 to 62.9, but adding it to URA+CPIA raises AP from 64.1 to 65.3.Source: §E.1 · p. 16

E.2 HYPERPARAMETER SENSITIVITY

  • On V2X-Seq, increasing K from 5 to 20 improves AP/AMOTA from 53.6/58.7 to 54.6/61.2; removing the cap adds no further gain over default budgets.On Griffin-25M, AP and AMOTA improve as K increases from 10 to 50, while IDS falls from 55 to 50.Source: §E.2 · p. 17
  • Enabling learned association lowers IDS from 76/63 to 70/50 on V2X-Seq/Griffin-25M, with AMOTA changing from 60.5/68.7 to 61.1/68.8.Increasing the association radius to 3.0 m lowers AMOTA to 59.3/67.9 and does not consistently improve the metrics.Source: §E.2 · p. 17

E.3 DETAILED DESIGN ABLATIONS

Detailed ablations test admission timing, cooperative feature refinement, and identity reuse; a separate compression evaluation measures payload savings alongside accuracy.

E.3 DETAILED DESIGN ABLATIONS

  • On the fixed V2X-Seq model, moving admission before temporal update reduces AP/AMOTA by 4.3/10.2 points; raising τCSA from 2.2 to 3.0 m reduces them by 0.3/1.1 points.The model is more sensitive to the tested timing change than to the tested spatial-threshold change.Source: §E.3 · p. 17
  • Removing local readout reduces AP/AMOTA from 54.8/61.1 to 52.3/54.7 and raises IDS from 70 to 97; mean-vector features yield 51.2/52.9 and 163 IDS.These retrained controls support instance-specific cooperative semantics and local context for refinement.Source: §E.3 · p. 17
  • Map-only yields AP/AMOTA of 54.7/60.5 and 76 IDS, close to Full’s 54.8/61.1 and 70; association-only lowers AMOTA to 32.5 and raises IDS to 167.Full adds 0.6 AMOTA points under this protocol by associating unmapped queries with existing tracks and recording accepted correspondences; Figure 9 compares Full and sticky-ID across five frames.Source: §E.3 · pp. 17–18, Figure 9 · p. 18

F.1 LATENCY INJECTION PROTOCOL

  • The Griffin-25M latency test selects drone observations 0, 100, or 200 ms earlier from a 10 Hz stream for evaluation at 2 Hz.Cooperative centers are compensated in the drone frame, transformed to the ego frame, then integrated; V2X-Seq calibration perturbations separately test fixed yaw and translation offsets.Source: §F.1 · p. 18

F.2 CODEBOOK COMPRESSION

  • The default codebook reduces estimated steady-state payload by approximately 95.0%, with AP/AMOTA decreasing by 0.2/0.1 points and IDS remaining at 50.Reducing Kcode to 256 lowers AP/AMOTA to 64.3/66.2; a single codeword yields 63.3/65.7.Source: §F.2 · p. 19, Table 15 · p. 19

G REPRODUCIBILITY AND EVALUATION PROTOCOL

The paper uses official validation splits and benchmark evaluators, with reported model, training, inference, routing, and communication-accounting settings specified for reproducibility.

Evaluation Protocols

  • Evaluation uses official validation splits; unless stated otherwise, it covers cars within 50 m, with AP averaged over center-distance thresholds of 0.5, 1, 2, and 4 m.AP, AMOTA, and MOTA are percentages; IDS is a count, and communication rates are in B/s.Source: G REPRODUCIBILITY AND EVALUATION… · p. 19

Architecture and Training

  • The model uses a ResNet-50, four-level FPN, and six-layer decoder with C = 256; the backbone and FPN are frozen while cooperative modules and decoder layers are trained.Training uses AdamW, learning rate 3 × 10^-4, weight decay 0.01, batch size 8, cosine annealing, and mixed precision on four GPUs.Source: G REPRODUCIBILITY AND EVALUATION… · p. 19

Query Banks and Cooperative Updates

  • V2X-Seq uses 360 slots, 120 ego-reserved slots, and admission budget K = 35; Griffin-25M uses 900, 300, and K = 50, respectively.CSA follows the first decoder layer and temporal update; both Full configurations use local readout, and CPIA supervision targets temporally retained live ego tracks.Source: G REPRODUCIBILITY AND EVALUATION… · pp. 19–20

Output Routing and Identity Management

  • Aligned queries with valid ego-frame projections enter CSA/URA; other valid queries follow a separate box-output path, and both paths reuse valid identity mappings.Mapping reuse requires confidence of at least 0.1, a live unclaimed ego ID, and an unexpired mapping; temporal banks and identity state reset before each sequence.Source: G REPRODUCIBILITY AND EVALUATION… · p. 20

Ablation Protocols

  • Each ablation variant completes cooperative training from its own preceding model and is run three times; reported metrics come from the run with median validation AMOTA.Hyperparameter, compression, and CSA/CPIA interventions use Full weights and change only specified inference conditions; local-readout and mean-vector controls are retrained.Source: G REPRODUCIBILITY AND EVALUATION… · p. 20

Communication and Codebook Synchronization

  • Communication accounting counts transmitted queries before receiver-side filtering or admission, excludes packet headers, and reports query payload separately from any one-time codebook transfer.V2X-Seq’s approximately 80 queries per frame at 2 Hz yield 8480 B/s with Kcode = 256; Griffin-25M’s approximately 62.4 queries per drone frame at 10 Hz yield 3.37×10^4 B/s with Kcode = 512.Source: G REPRODUCIBILITY AND EVALUATION… · p. 20

H METHOD NOTES AND LIMITATIONS

FFBL-Coop separates cooperative query placement from ego-track identity inheritance, rather than letting one match govern both. Its stated boundaries include dependence on cooperative evidence, robustness to calibration and latency, and evaluation focused primarily on cars.

H METHOD NOTES AND LIMITATIONS

  • FFBL-Coop separates remote-evidence allocation from identity inheritance: CSA allocates queries, URA refines them with ego observations, and CPIA resolves identities using persistent mappings.Its distinction is separating query placement from identity inheritance within the recurrent pipeline, not introducing unmatched-query retention or ego-image refinement as new capabilities.Source: H METHOD NOTES AND LIMITATIONS · pp. 20–21
  • Performance depends on cooperative coverage, detection quality, and calibration; out-of-region concatenation retains localization errors, and persistent mappings may propagate accepted identity errors.Evaluation primarily concerns cars; broader category coverage remains to be validated.Source: H METHOD NOTES AND LIMITATIONS · p. 21
  • 54.8/61.1 to 49.4/60.4 AP/AMOTA on V2X-Seq under a 1° yaw perturbation; the robustness evaluation covers the complete pipeline.The evaluation does not isolate individual robustness mechanisms.Source: H METHOD NOTES AND LIMITATIONS · p. 21
  • At 200 ms latency, Griffin-25M AP decreases from 65.3 to 59.4, while AMOTA changes from 68.8 to 68.4 with velocity compensation.Reported query payload rates exclude packet headers and codebook synchronization; deployment traffic also depends on update frequency and protocol overhead.Source: H METHOD NOTES AND LIMITATIONS · p. 21
  • Less reliable calibration, variable delays, packet loss, and identity coordination across multiple agents remain future-work directions.These are identified as directions for future work rather than evaluated settings.Source: H METHOD NOTES AND LIMITATIONS · p. 21

Related papers

Loading 2610.01750v1…