PaperlayerView exact revision on arXiv

Source-linked AI summary

Decentralized Multi-View, Multi-Zone Tracking for Scalable Camera Networks

David Voihanski, Mor Sinai, Ben Zion Bobrovsky

arXiv:2610.33069v1cs.CV

TL;DR

Centralized multi-camera tracking can struggle to scale because of bandwidth and processing bottlenecks. This paper distributes tracking across dynamic BEV zones and uses decentralized coordination to merge tracks across zone boundaries. Increasing delay from 0 to 5 s improves HOTA by 2.8 percentage points on Dataset A and 3.0 percentage points on Dataset B.

  • Problem

    Centralized multi-camera tracking faces bandwidth saturation and computational bottlenecks that limit scaling to large camera deployments.

    Source: §I · p. 1

  • Method

    The system assigns dynamic BEV zones to independent tracking instances and uses decentralized locks and periodic global collate to merge cross-zone tracks.

    Source: §I · p. 1

  • Results

    Increasing delay from 0 to 5 s improves HOTA by 2.8 percentage points on Dataset A and 3.0 percentage points on Dataset B, while reducing IDSW from 13 to 4 and from 43 to 26, respectively.

    Source: B. Component Ablation and Sensitivity… · p. 6

  • Takeaways & Limitations

    Compared with a single-instance multi-view tracker, the architecture trades a small amount of tracking quality for scaling across massive deployments with bounded per-device load.

    Source: §VIII · p. 8

    At 40 cameras, the scale-out experiment uses generated trajectories, leaving end-to-end tracking accuracy at that scale unverified.

    Source: §VII · p. 8

Abstract

from arXiv · show

We present a decentralized multi-view tracking system for scalable camera networks. Single-view detections are projected to bird's-eye-view (BEV) coordinates and fused by independent multi-view trackers, each managing a dynamic spatial zone. To support scalability across spatially partitioned camera networks, we introduce a decentralized layer for trajectory unification, utilizing a locking protocol and periodic track-merging operations termed global collate. This process is supported by a delayed-output buffering mechanism that retroactively applies history corrections. The configurable delay window absorbs asynchronous coordination latency and helps stabilize cross-zone identity consistency. We detail the design of dynamic zone polygons, consensus-based coordination, and the validation framework, which together maintain global ID consistency with minimal accuracy loss compared to a single-instance baseline in the evaluated configurations.

I. Introduction

Centralized multi-camera tracking faces scaling limits from bandwidth and processing demands; this paper organizes existing tracking components into a decentralized architecture for bounded per-instance computation and cross-zone identity consistency. It builds on BEV fusion and location-aware matching,,,, while distributing coordination across independent instances.

I. Introduction

  • The architecture assigns dynamic BEV zones to independent instances and coordinates cross-zone identity through periodic global collate.The paper frames its contribution as organizing existing components, not proposing a new single-camera tracker, BEV projection, consensus protocol, or assignment solver.Source: §I · p. 1
  • Dynamic zone polygons and a decentralized overlap map support neighbor discovery as camera footprints are partitioned across instances.These mechanisms are listed as specific technical contributions.Source: §I · p. 1
  • Decentralized locks, hit-ratio gating, and force-threshold collision resolution form the global-collate pipeline for unifying cross-zone trajectories.The architecture also includes validation and reversion with delayed output to stabilize IDs.Source: §I · p. 1

II. Related Work

  • Centralized systems face bandwidth and compute bottlenecks, while prior multi-view work uses BEV fusion, or location-aware city-scale matching,.Single-camera tracking commonly combines detection with Kalman filtering,; centralized scalability concerns are also discussed in,.Source: §II · p. 1
  • The system builds on per-camera smoothed leg points, local BEV fusion, and Raft-based replication and locking among independent instances.The coordination layer is described as eventually consistent without a single point of failure or large central node.Source: §II · pp. 1–2

III. Problem Setting and System Assumptions

The system uses a three-layer tracking hierarchy, with dynamic BEV zones and periodic cross-zone track merging in its decentralized layer. A validation and reversion module revisits merge decisions using accumulated history.

III. Problem Setting and System Assumptions

  • Per-camera trackers provide smoothed BEV leg coordinates to local multi-camera instances, which fuse detections before decentralized coordination.The decentralized layer uses dynamic zones and global collate to coordinate tracks crossing zone boundaries.Source: §III · p. 2
  • Each zone polygon begins around early detections and expands as new detections arrive outside its current footprint.After sufficient coverage, the polygon stabilizes as the zone’s spatial footprint in bird’s-eye coordinates.Source: Figure 3 · p. 2
  • An incoming BEV detection outside a zone expands its polygon and updates the decentralized map; intersections identify neighboring zones for later global collate.The system revisits merge decisions with accumulated history to correct errors before they persist in output.Source: Figure 4 · p. 2, §III · p. 2

A. Zone Polygon and Overlap Map

Zones grow to cover observed BEV tracks, and their intersections identify neighbors for decentralized coordination. Periodic, locked global collate matches tracks across those boundaries, reducing trajectory fragmentation.

A. Zone Polygon and Overlap Map

  • An out-of-polygon track center triggers re-estimation of the zone as the concave hull of accumulated points, allowing the footprint to grow with scene exploration.Zone polygons are synchronized through a Raft-supported decentralized dictionary.Source: A. Zone Polygon and Overlap Map · p. 2

2) Global Lock:

  • Instances discover candidate neighbors by checking geometric intersections between their zones, then protect zone-pair matching with decentralized locks.Locks represent zone pairs, such as 1_2, and support blocking acquisition.Source: 2) Global Lock · p. 3

B. Decentralized Infrastructure

  • A local dictionary and lock manager are maintained through distributed consensus, while persistent RPC channels connect each instance to overlapping neighbors.The channels support asynchronous collate communication without repeated connection setup.Source: B. Decentralized Infrastructure · p. 3

C. Global Collate Protocol and Locking

  • Without cross-zone matching, boundary-crossing trajectories fragment into distinct local IDs; global collate unifies them across zones.The mechanism periodically coordinates localized tracking with global matching across overlapping zones.Source: C. Global Collate Protocol and Locking · p. 3, Figure 5 · p. 3

1) Execution Gating and Timers:

Global matching runs only when quorum and active neighbors are present, with timers governing merge attempts and validation. After locks are acquired, instances exchange track histories and match them using gated kinematic and appearance costs.

1) Execution Gating and Timers:

  • Global matching is gated on a valid quorum and active neighbors; collate and validation timers determine when synchronization cycles begin.If lock acquisition fails within the timeout, the instance releases held locks and skips the cycle, ensuring only one instance modifies shared state in an overlapping region at a time.Source: 1) Execution Gating and Timers · p. 3

3) Collate Protocol and Matching:

  • After acquiring locks, instances send recent confirmed histories to neighbors and align them temporally before computing pairwise association costs.The exchanged histories include timestamps, Kalman states, and embeddings.Source: 3) Collate Protocol and Matching · p. 3
  • The matching cost combines a spatial kinematic metric with a visual appearance metric, balancing the two through λ.The spatial metric is Mahalanobis distance using the local track’s projected Kalman covariance; appearance consistency uses Euclidean distance between visual Re-ID features.Source: 3) Collate Protocol and Matching · p. 3
  • Pairs exceeding either the maximum Mahalanobis distance or maximum feature distance receive infinite cost, and the Hungarian algorithm selects track pairs for relabeling.Accepted matches are applied by updating local track IDs and logging the merge timestamp for future validation.Source: 3) Collate Protocol and Matching · p. 3

D. Validation and Reversion

Proposed cross-zone merges are screened using accumulated association evidence and consistency costs. Accepted merges remain provisional so later validation can trigger a swap or reversion when evidence favors another identity.

D. Validation and Reversion

  • Proposed merges accumulate association hits and matching costs, and must pass a minimum hit-ratio gate before they can proceed.A hit is a detection successfully associated with a track during a tracking update.Source: D. Validation and Reversion · p. 3
  • When an incoming identity already exists locally, a forced swap is allowed only if its normalized consistency cost is below the force threshold and lower than the current-ID cost.The normalized cost uses accumulated matching cost, supporting hits, and a hit-count normalization parameter; lower values indicate better consistency.Source: D. Validation and Reversion · pp. 3–4
  • Recently merged IDs are re-evaluated over time; a consistently better candidate can trigger a swap, while collisions attempt reversion and may remove the conflicting track if reversion fails.The validation flow compares the validated costs of the old and candidate tracks before reverting a match or swapping the local ID and updating the matches database.Source: D. Validation and Reversion · p. 4, Figure 7 · p. 4

E. Output Delay Buffer and History Fix-up

A temporal buffer and history-fix layer absorb asynchronous coordination overhead by delaying official output while retroactively applying tracking corrections. The configured delay reduces transient identity flicker at zone boundaries, with accuracy benefits saturating at about 3.0 seconds.

E. Output Delay Buffer and History Fix-up

  • A temporal buffer and history-fix layer manage asynchronous global operations and network coordination overhead.Track states and match hypotheses are queued with timestamps in a fixed-length FIFO buffer, then corrected retroactively before release.Source: E. Output Delay Buffer and History… · p. 4
  • Without buffering, provisional local IDs can reach live output before asynchronous RPC coordination returns, producing transient identity flicker at zone boundaries.The response delay can be configured down to zero seconds, but operating without a buffer exposes the system to this flicker.Source: E. Output Delay Buffer and History… · p. 4
  • Accuracy benefits from buffering saturate at approximately 3.0 s, while the primary 5.0 s operating point provides a conservative, flicker-free identity stream.The routine applies ID remappings, modifies frames using newly confirmed tracks, and executes merges, forced swaps, or validation-driven reversions before buffered output is released.Source: E. Output Delay Buffer and History… · p. 4

V. Experimental Setup

The evaluation uses proprietary retail data because existing public benchmarks have constraints that limit assessment of decentralized coordination and long-term cross-zone consistency. The two fixed-camera datasets are time-synchronized and annotated with global identities.

A. Dataset Selection and Rationale

  • Public benchmarks have evaluation constraints: AI City Challenge combines domain shifts with a single averaged score, MMPTrack withholds test labels, and WILDTRACK spans only 400 frames.These constraints limit pedestrian-specific analysis, independent cross-zone ID analysis, or sustained stress-testing of locking and drift, respectively.Source: A. Dataset Selection and Rationale · p. 4
  • The quantitative evaluation therefore uses large-scale proprietary internal data intended to reflect real-world deployment conditions and assess global ID stability and decentralized coordination.The raw videos and annotations cannot be publicly released for privacy and business reasons.Source: A. Dataset Selection and Rationale · p. 5, B. Proprietary Datasets · p. 5

B. Proprietary Datasets

  • Both retail datasets use fixed, time-synchronized cameras recording at 5 FPS, with ground-truth identities consistent across cameras and zones.The annotations assign each person one global GT track ID across all observed views.Source: B. Proprietary Datasets · p. 5
  • Dataset A is a supermarket recording with 11 cameras, 62 global tracks, and 16,854 labeled detections; Dataset B is a general store with 8 cameras, 45 tracks, and 14,943 detections.The recordings span 2:20 and 2:50 minutes, respectively.Source: B. Proprietary Datasets · p. 5

A. Accuracy Evaluation

The decentralized system retains most tracking identity quality, with accuracy trade-offs increasing as deployments are split into more zones. Global collate provides the largest IDSW reduction, while delayed history fix-up further stabilizes output.

A. Accuracy Evaluation

  • 1.4% lower HOTA and 0.9% lower IDF1 on Dataset A’s two-instance run accompany unchanged IDSW (4); Dataset B is flat in HOTA, slightly higher in IDF1, and rises from 24 to 26 IDSW.The accuracy comparison is at a 5.0 s response delay.Source: A. Accuracy Evaluation · p. 5, Table I · p. 5
  • Approximately 2.6 percentage points lower HOTA and IDSW rising from 4 to 8 on Dataset A’s three-instance run accompany only 3–4 cameras per instance and restricted local view redundancy.The additional identity errors are attributed to intra-zone tracking rather than boundary coordination failures.Source: A. Accuracy Evaluation · p. 5, Table I · p. 5

B. Component Ablation and Sensitivity Analysis

  • IDSW falls from 48 to 13 on Dataset A and from 64 to 43 on Dataset B with global collate, then falls to 4 and 26 with delayed history fix-up.Global collate is enabled even in the zero-delay setup; delayed history fix-up retroactively applies merge and reversion decisions.Source: B. Component Ablation and Sensitivity… · pp. 5–6, Table II · p. 5
  • 2.8 percentage points higher HOTA on Dataset A and 3.0 percentage points higher HOTA on Dataset B result from increasing delay from 0 to 5 s; gains saturate at approximately 3 s.The 5 s operating point is conservative rather than necessary for all deployments.Source: B. Component Ablation and Sensitivity… · p. 6, Table III · p. 6

C. Boundary Traffic and Cross-Zone Attribution

Across evaluated configurations, the matching loop caused zero boundary identity switches, even under substantial boundary traffic. More zones reduced total latency in the tested worst-case setting, while scale-out keeps instance workloads bounded but requires additional hardware and retains accuracy trade-offs.

C. Boundary Traffic and Cross-Zone Attribution

  • 34 ground-truth trajectories crossed zone boundaries 165 times in the three-instance Dataset A split, spending an average of over 1,100 frames in overlap regions.This is a highly active boundary-traffic setting.Source: C. Boundary Traffic and Cross-Zone… · p. 6, Table V · p. 7
  • Zero boundary identity switches were attributed to the matching loop across evaluated configurations; remaining errors were missed associations or intra-zone failures, not destructive matching-loop changes.On Dataset B, even missed boundary associations were eliminated despite 59 crossings.Source: C. Boundary Traffic and Cross-Zone… · p. 6, Table VI · p. 7

D. Computational Scalability and Trade-offs

  • Each instance’s computational input depends on its assigned zone rather than total camera count, provided zone camera counts and neighboring overlaps remain bounded; adding zones requires additional hardware.The scale-out interpretation is architectural, and practical limits include infrastructure, network capacity, synchronization reliability, and overlap density.Source: D. Computational Scalability and… · pp. 6–7
  • Total latency decreased as zones were added with 40 cameras at 100% overlap; core tracking time fell faster than decentralized coordination overhead rose.The figures show the total-latency trend and its tracking-versus-coordination breakdown.Source: D. Computational Scalability and… · pp. 6–7, Figure 8 · p. 7, Figure 9 · p. 7
  • Neighbor-to-neighbor association introduces multi-hop fusion, temporal lag, and missed boundary merges when evidence is insufficient; conservative thresholds reduce erroneous merges but can leave tracks unmatched.Overlap regions are sized to provide sufficient temporal evidence for cross-zone association.Source: D. Computational Scalability and… · p. 8

VII. Limitations

The evaluation of the system is bounded by physical deployment scale and data availability. Its tracking model also requires continuous, intersecting zone boundaries.

System Limitations

  • The largest scale-out experiment covers 40 cameras using generated trajectories for latency and throughput, leaving end-to-end tracking accuracy at that scale unverified.Real-data physical evaluation reaches at most three decentralized instances; testing synchronization across larger, more complex layouts remains an engineering challenge.Source: §VII · p. 8

Data Constraints

  • Annotated real-world sequences cover 8 to 11 cameras for under 3 minutes because global multi-camera identity labeling is resource-intensive.Source: §VII · p. 8
  • The system depends on continuous, intersecting zone boundaries and cannot inherently bridge disjoint streams or isolated regions without an auxiliary wide-area re-identification layer.Source: §VII · p. 8

VIII. Conclusion

The system partitions the bird’s-eye plane into dynamic zones managed by independent multi-view trackers and coordinates cross-zone track unification. Its reported findings include clean merges, no lock-acquisition or communication failures under contention, and a small tracking-quality trade-off against a single-instance tracker.

VIII. Conclusion

  • The system partitions the bird’s-eye plane into dynamic zones, each handled by an independent multi-view tracker, and coordinates cross-zone unification through decentralized maps and locks.Source: §VIII · p. 8
  • Compared with a single-instance multi-view tracker, the architecture trades a small amount of tracking quality for scaling across massive deployments with bounded per-device load.It distributes workload and removes reliance on high-end centralized servers.Source: §VIII · p. 8
  • The global collate mechanism achieves dozens of clean cross-zone merges, with zero lock-acquisition or communication failures under contention.Validation and reversion modules correct minor edgecase anomalies while preserving tracking continuity across instances.Source: §VIII · p. 8
Loading 2610.33069v1…