Source-linked AI summary
GRACE: Geometry- and Ray-Aware Camera-Efficient Multi-View Pedestrian Tracking
Taigo Sakai, Hiroki Kouno, Naoki Kato, Kazuhiro Hotta
TL;DR
Fewer cameras reduce deployment costs but also remove views that can correct BEV localization errors, while temporary score drops can split tracks. GRACE combines volumetric and homography-based fusion, viewing-direction conditioning, and track recovery; with two cameras it improves MOTA over TrackTacular on WildTrack and MultiviewX.
Problem
With fewer cameras, BEV responses displaced from pedestrians' true positions are less likely to be corrected by another view, and temporary detection-score drops can split existing tracks.
Source: §1 · p. 1, §2.1 · p. 1Method
GRACE fuses homography-based BEV features with volumetrically lifted features, supplies camera viewing directions, and uses low-confidence detections only to continue existing tracks.
Source: §1 · p. 1Results
Takeaways & Limitations
Abstract
from arXiv · showhide
Reducing the number of cameras reduces the deployment cost but removes views that correct BEV responses stretched away from true pedestrian positions by projection and short score drops that can split tracks} in Bird's-Eye View (BEV) tracking. We introduce GRACE, a camera-efficient multi-view tracker with three components. Volumetric-Guided Fusion combines homography-based BEV features with features lifted through 3D space. Ray Conditioning exposes each camera's viewing direction to the fusion network. Its tracking component, BEV Track Recovery (BTR), uses low-confidence detections only to continue existing tracks. The same detections cannot start new tracks. With two WildTrack cameras, GRACE improves MOTA from 83.54 for TrackTacular, our baseline, to 91.07.
1. INTRODUCTION
Few-camera BEV tracking is vulnerable to directional projection errors and temporary score drops that can split identities. GRACE combines geometry-aware fusion and track recovery, improving two-camera WildTrack MOTA over TrackTacular.
- Few-camera BEV tracking faces directional projection errors that displace responses and brief confidence drops that can split one track into two identities.Source: §1 · p. 1
- GRACE combines VGF, Ray Conditioning, and BTR: fusion balances homography and volumetric features, rays encode viewing direction, and BTR permits low-confidence detections to continue tracks but not start them.Source: §1 · p. 1
- TrackTacular is the primary baseline because it also performs detection and tracking in BEV.Source: §1 · p. 1
- 91.07 MOTA on two WildTrack cameras exceeds two-camera TrackTacular's 83.54 MOTA; GRACE also reaches 79.36 MOTA on MultiviewX.Source: §1 · p. 1
- The proposed geometry-aware fusion and recovery use only camera calibration and past observations.Source: §1 · p. 1
2. THE PROPOSED METHOD: GRACE
GRACE combines Volumetric-Guided Fusion, Ray Conditioning, and BEV Track Recovery to address projection errors and track splits in few-camera BEV tracking. Its WildTrack comparison reports higher MOTA than two-camera TrackTacular with the same camera count.
2.1. Problem Formulation
- With few cameras, fewer views can correct localization errors, while temporary confidence drops can split an existing track and assign a new identity.VGF and Ray Conditioning address projection errors; BTR addresses track splits.Source: §2.1 · p. 1
2. THE PROPOSED METHOD: GRACE
- 91.07 MOTA with two WildTrack cameras, versus 83.54 for two-camera TrackTacular; the seven-camera reference reaches 90.73 MOTA.The seven-camera reference uses VGF + Ray detections with the TrackTacular tracker.Source: Figure 1 · p. 2
2.2. Volumetric-Guided Fusion
- Volumetric-Guided Fusion mixes homography-projected features with features lifted through calibrated 3D space, using a learned gate and camera coverage.The coverage prior increases volumetric weight where few cameras can provide mutual correction; the learned term adapts the balance to observed features.Source: §2.2 · pp. 1–2
2.3. Ray Conditioning
- Ray Conditioning supplies each camera’s viewing direction at BEV locations, distinguishing locations with equal camera counts but different viewing directions.The two-channel direction map is computed from calibration and concatenated with each camera’s volumetric feature.Source: §2.3 · p. 2
2.4. BEV Track Recovery
- BEV Track Recovery matches low-confidence detections to existing tracks but bars them from initializing tracks, and predicts through missing detections for at most two frames under specified conditions.The low-confidence matching threshold is τ − δ with δ = 0.2; prediction is allowed when prior confidence is at least 0.4, and BTR remains online.Source: §2.4 · p. 2
3. EXPERIMENTAL SETUP
Experiments use two-camera settings on WildTrack and MultiviewX, with camera pairs selected by calibration and a separate WildTrack study spanning additional camera counts.
3. EXPERIMENTAL SETUP
- All methods use WildTrack C1+C3 and MultiviewX C1+C6 on the datasets’ standard BEV evaluation regions.The same camera pair is used for every method on each dataset.Source: §3 · p. 2
3.1. Camera Configuration
- The two-camera setting is the minimum that enables multi-view fusion; the camera-count study also evaluates three to seven cameras.The study examines how the effects change as more views become available.Source: §3.1 · p. 2
- With two cameras, directional spreading is more visible in BEV responses, while additional views produce more concentrated responses.The WildTrack camera-count study reports BTR’s largest gain with two cameras.Source: Figure 3 · p. 3, Figure 6 · p. 4
- C1+C3 is the only WildTrack pair with complete BEV coverage; 44.98% of the grid is visible to both cameras.For MultiviewX C1+C6, at least one camera covers 95.62% of the evaluation area and both cameras see 44.21%.Source: §3.1 · p. 3, Figure 5 · p. 3
- WildTrack experiments fix C1+C3 across two-camera runs, follow TrackTacular preprocessing and ResNet-18 training, and share input processing, detection threshold, and evaluator across framework methods.The camera pair is selected using calibration before model evaluation.Source: §3.1 · p. 3
4. RESULTS
GRACE improves two-camera tracking on WildTrack and MultiviewX; camera-count and component studies examine where its fusion, ray conditioning, and recovery gains arise.
4.1. WildTrack Results
- With the same C1+C3 input, GRACE improves WildTrack results over TrackTacular from 82.53 to 86.38 MODA and from 83.54 to 91.07 MOTA.CaMuViD and RMCS are detection-only methods, so their tracking cells are omitted; the seven-camera block compares TrackTacular and GRACE.Source: §4.1 · p. 4
4.2. Camera-Count Study
- Across two to seven WildTrack cameras, VGF + Ray raises MODA from 86.38 to 92.58, versus 83.54 to 88.50 for TrackTacular.BTR adds 3.43 MOTA points with two cameras but at most 0.76 with three to seven; parameters remain fixed.Source: §4.2 · p. 4
4.3. MultiviewX Results
- With two cameras, GRACE reaches 79.36 MOTA on MultiviewX, 11.09 points above TrackTacular; MODA rises from 71.26 to 77.09 and IDF1 from 67.34 to 75.05.The two-camera comparison uses the same C1+C6 input for all evaluated methods.Source: §4.3 · p. 4, Table 2 · p. 4
4.4. Component Contributions
- On WildTrack C1+C3, VGF improves MODA by 2.52, IDF1 by 7.43, and MOTA by 3.05 over TrackTacular; Ray Conditioning then improves MODA by 1.33 and Recall by 1.54.The reported precision reduction is 0.19, described as small relative to the recall gain.Source: §4.4 · p. 4, Table 3 · p. 4
- BTR leaves detections unchanged but raises IDF1 by 2.83 and MOTA by 3.43 over VGF + Ray, separating temporal association gains from spatial-evidence changes.The ablation is on WildTrack C1+C3.Source: §4.4 · p. 4, Table 3 · p. 4
- At seven cameras, BTR changes MOTA from 90.73 to 91.04, compared with its 3.43-point gain at two cameras; overlapping views already reduce short detection gaps.Source: §4.4 · p. 4
4.5. Effect of Ray Conditioning
- With the same two WildTrack cameras, TrackTacular produces elongated BEV activations around pedestrians, while GRACE produces more compact activations.The comparison attributes reduced directional spreading in this setting to Ray Conditioning.Source: §4.5 · p. 4, Figure 7 · p. 4
5. CONCLUSION
GRACE targets projection errors and short detection gaps with VGF, Ray Conditioning, and BTR. It improves two-camera tracking over TrackTacular on WildTrack and MultiviewX.
- With two WildTrack cameras, GRACE reaches 91.07 MOTA, 1.36 points higher than TrackTacular with seven cameras.Source: §5 · pp. 4–5
- VGF and Ray Conditioning address projection errors, while BTR continues an existing track when detection confidence temporarily falls below the normal threshold.Source: §5 · p. 4
- On MultiviewX, GRACE improves MOTA by 11.09 points over TrackTacular using the same two cameras.Source: §5 · p. 5