PaperlayerView exact revision on arXiv

Source-linked AI summary

ARC-SORT: Adaptive, Robust, Compensated Multi-Object Tracking with Interacting Multiple Models

Girish Kumar Adari

arXiv:2610.22306v1cs.CV

TL;DR

Sports broadcasts combine abrupt athlete motion with camera pans and zooms, challenging tracking-by-detection association. ARC-SORT uses predictive uncertainty to adapt matching and defer close identity decisions, and drops motion gating for long-gap appearance linking. It reports 80.31 HOTA on Deep-EIoU’s public detections and embeddings versus 77.20, and 81.82 HOTA on SportsMOT test with its own detector stack.

  • Problem

    Sports broadcasts combine fast, abrupt athlete motion that violates constant-velocity assumptions with simultaneous camera pans and zooms.

    Source: §1 · p. 1

  • Method

    ARC-SORT uses IMM covariance to adapt matching tolerance, assignment margins to defer ambiguous identity commitments, and appearance-only linking when long-gap motion information has run out.

    Source: §1 · p. 1

  • Results

    80.31 HOTA on Deep-EIoU’s public detections and embeddings versus its published 77.20, while ARC-SORT’s own detector stack reaches 81.82 HOTA on SportsMOT test.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ARC-SORT applies tracker uncertainty to association tolerance and commitment, and removes the motion gate for long-gap appearance linking.

    Source: §1 · p. 2

    Deferred commitment is pairwise and single-swap, its gain tracks the embedding stack, the long-horizon stage is offline, and public model-zoo detection caps HOTA.

    Source: §5 · p. 8

Abstract

from arXiv · show

Multi-object tracking in sports broadcasts is hard for two reasons: athletes move fast and change direction abruptly, and the camera pans and zooms at the same time. We present ARC-SORT (Adaptive, Robust, Compensated SORT), a tracking-by-detection tracker that trains nothing and runs in real time. Its design rule: the tracker measures its own uncertainty, and that measurement decides how it acts. An Interacting Multiple Model (IMM) Kalman filter gives every track a predictive covariance, and we turn it into a per-track search region: the less reliable the motion prediction, the larger the region in which a detection may match (uncertainty-adaptive Expansion IoU, EIoU). On identical inputs, this continuous coupling is what matters: the same covariance used as a hard Mahalanobis gate hurts at every threshold we swept, and richer covariance summaries add nothing measurable. When two tracks cross, the assignment cost itself reports whether the decision was close. ARC-SORT keeps such pairs under review and applies at most one corrective identity swap once post-crossing appearance evidence shows the first decision was wrong; this adds +1.0 HOTA (higher order tracking accuracy) on both the SportsMOT and DanceTrack test sets at under a millisecond per frame. For players who leave the frame for seconds, motion extrapolation carries no information, so our offline re-identification drops the usual motion gate and links tracklets by appearance alone. ARC-SORT reaches 81.82 HOTA on the SportsMOT test set with the fewest identity switches of the 69 leaderboard entries as of August 2026 and tied-third multi-object tracking accuracy (MOTA, 97.09), and 62.1 HOTA on DanceTrack test with the same association settings. On Deep-EIoU's public detections and embeddings it reaches 80.31 HOTA, +3.1 over Deep-EIoU. It runs at about 50 frames per second on a laptop RTX 3060 GPU. Code will be released.

1. Introduction

ARC-SORT addresses fast, nonlinear athlete motion and simultaneous camera movement by letting the tracker’s uncertainty govern association and identity commitment. Its measured benchmark results and controlled comparisons support covariance-adaptive matching, deferred identity decisions, and appearance-only linking across long gaps.

Motivation

  • Sports broadcasts combine abrupt athlete motion that violates constant-velocity assumptions with panning and zooming cameras.Existing responses include fixed, scheduled, or velocity-based box expansion and domain-trained motion; covariance and assignment-margin signals are often discarded or used as a hard veto.Source: §1 · p. 1

Contributions

  • ARC-SORT uses IMM predictive covariance to continuously expand each track’s matching region, while assignment margins flag close crossings for later identity correction.At multi-second out-of-view gaps, its offline re-identification drops the motion gate and links tracklets by appearance alone.Source: §1 · pp. 1–2

Results

  • +1.0 HOTA on both SportsMOT and DanceTrack test sets comes from deferred identity commitment; removing the merge’s motion veto adds +4.9 HOTA on SportsMOT val.These measurements compare otherwise identical systems; the merge result also transfers to DanceTrack under a different embedding model.Source: §1 · p. 2

Contributions

  • A controlled comparison finds hard Mahalanobis gating with the same covariance degrades association at every swept threshold, while richer covariance uses add nothing measurable.The scalar relative positional uncertainty is empirically sufficient for setting association tolerance.Source: §1 · p. 2

Results

  • 80.31 HOTA on Deep-EIoU’s public detections and embeddings versus its published 77.20, while ARC-SORT’s own detector stack reaches 81.82 HOTA on SportsMOT test.The SportsMOT result has the fewest identity switches among 69 leaderboard entries and tied-third MOTA at 97.09.Source: §1 · p. 2

2. Related Work

Related work strengthens tracking-by-detection through appearance, recovery, camera compensation, and learned association, but commonly leaves association uncertainty unused or treats it as a hard gate. ARC-SORT instead couples uncertainty to overlap tolerance, defers close identity decisions, and removes motion constraints from long-gap appearance linking.

Tracking-by-detection

  • SORT-family overlap pipelines generally ignore predictive covariance or use it as a hard Mahalanobis veto, while the association cost remains uncertainty-blind and matches are committed immediately [7] [42].IMM covariances have long been used in radar tracking, where PDAF/JPDAF incorporate them continuously in soft filter updates.Source: §2 · p. 2

Expanded and buffered matching

  • Prior expanded-box methods use fixed factors or discrete velocity switches, whereas ARC-SORT sets overlap tolerance continuously from predictive covariance without training [43], [20], [16], [40].Probabilistic-IoU methods score overlap between distributions, and UTrack propagates detector uncertainty, unlike ARC-SORT’s use of motion-model uncertainty [25] [41],.Source: §2 · p. 2

Deferred data association

  • MHT and PDAF/JPDAF defer or soften ambiguity, but SORT-lineage trackers generally commit at crossings and leave revision to offline refinement, [5, 13], [12] [38].ARC-SORT uses the assignment’s permutation margin to select rare ambiguous pairs for bounded, two-hypothesis revision within the causal stream.Source: §2 · p. 2

Learned motion and benchmarks

  • ARC-SORT adapts through its filter without per-domain training, unlike learned motion, association, and end-to-end tracking approaches [21] [22] [27] [31], [2], [1] [36], [30], [14] [46].SportsMOT combines nonlinear athlete motion and panning cameras, while DanceTrack isolates nonlinear motion under a largely static camera [11], [39].Source: §2 · p. 3

Offline tracklet association

  • ARC-SORT removes the motion constraint from long-gap tracklet linking, unlike GTA’s offline relinking under appearance plus spatio-temporal constraints (81.0 on SportsMOT) [38].The paper identifies the motion constraint as the binding factor in its ablation.Source: §2 · p. 3

3. Method

ARC-SORT’s online tracker uses motion, camera compensation, and association evidence to manage uncertainty, while its offline stage links tracklets across long gaps by appearance alone. Its components include an IMM motion model, uncertainty-adaptive overlap, staged association, and bounded correction of ambiguous crossings.

3.1. Online tracker overview

  • The online overview predicts and warps tracks before association, then uses assignment margins to manage entanglement; corrections relabel tracks without changing filter states.The tracker also creates tracks from unmatched high-confidence detections and removes tracks lost over 30 frames.Source: §3.1 · p. 3, §3 · p. 3

3.2. IMM motion model

  • The IMM combines constant-velocity, constant-acceleration, and coordinated-turn models, with between-model disagreement contributing to the combined covariance.The models use box measurements; the coordinated-turn model uses an EKF, and its turn rate reduces to constant velocity when |ω| < 10^-6.Source: §3.2 · p. 3

3.3. Camera motion compensation (CMC) and re-update

  • Camera-motion compensation estimates a per-frame similarity transform from sparse optical flow and warps each sub-model’s state and covariance.After re-finding a track, CMC-corrected re-update interpolates virtual observations in a frozen frame and maps them forward before replay; it activates when a gap-frame translation reaches 1 px.Source: §3.3 · pp. 3–4

3.4. Uncertainty-adaptive Expansion IoU

  • Uncertainty-adaptive Expansion IoU maps the positional diagonal of each track’s innovation covariance to a per-track expansion factor for matching.Both boxes expand symmetrically before IoU is computed; the factor ranges from 0.2 to 1.2, and the matching region covers two predicted-center standard deviations in units of box size.Source: §3.4 · p. 4

3.5. Association pipeline

  • Stage 1 matches high-confidence detections using expanded IoU, observation-centric momentum, and appearance, with appearance refining only spatially plausible matches.Stage 2 matches recently tracked targets to low-confidence detections by plain IoU, and Stage 3 uses last-observed boxes for spatially valid recovery.Source: §3.5 · p. 4

3.6. Deferred identity commitment

  • A low permutation margin marks a crossing pair as entangled; later embedding evidence can trigger at most one corrective identity swap per episode.The episode snapshots appearance prototypes before post-crossing updates, closes 30 frames after the last entanglement, and requires at least four evidence frames for a swap.Source: §3.6 · pp. 4–5

3.7. The alternatives: same signal, other couplings

  • ARC-SORT compares alternative couplings of the same per-track prediction and covariance, including hard Mahalanobis vetoes, fixed or speed-based expansion, anisotropic expansion, and expected DIoU.The alternatives are tuned at their own matching thresholds on identical inputs.Source: §3.7 · p. 5

3.8. Long horizon: appearance-only tracklet merge

  • ARC-SORT’s offline merge links tracklets separated by multi-second gaps using appearance prototypes alone, removing a motion veto that carries no information under camera panning.On SportsMOT-val football, 64% of identity switches follow gaps over 20 frames; 97% of those are full camera exits, with a median exit of 111 frames.Source: §3.8 · p. 5

4. Experiments

Experiments evaluate ARC-SORT on SportsMOT and DanceTrack, including controlled comparisons on fixed inputs. Results show gains from removing the long-gap motion gate and from deferred commitment, while the mechanism ladder favors continuous covariance-based tolerance.

4.1. Setup

  • Validation experiments use train-split-only YOLOX-X weights and byte-identical cached inputs; end-to-end throughput is ~50 FPS including JPEG decode on a 6 GB laptop RTX 3060.SportsMOT tuning uses 45 validation sequences, while DanceTrack transfer uses 25.Source: §4.1 · pp. 5–6

4.3. Comparison on DanceTrack test

  • On DanceTrack test, ARC-SORT reaches 62.1 HOTA; against Deep OC-SORT with the same detections and embeddings, it gains +0.8 HOTA, +2.4 AssA, and +5.0 IDF1.The configuration transfers without retuning; the authors do not claim DanceTrack state of the art.Source: Table 3 · p. 7, §4.3 · p. 7, §4.6 · p. 8

4.4. Mechanism ladder: how to spend S

  • On identical SportsMOT validation inputs, the mechanism ranking runs veto ≪ fixed < switched < σ-pure < continuous ≈ anisotropic ≈ σ-blend; richer variants add nothing measurable.The scalar max(σx/w, σy/h) is empirically sufficient for association tolerance, while hard-gate uses of covariance hurt.Source: Table 4 · p. 7, §4.4 · p. 7

4.6. Deferred commitment: measure first, then intervene

  • Deferred commitment adds +1.00 HOTA on each test set (80.82→81.82 SportsMOT; 61.1→62.1 DanceTrack), while corrective swaps increase the counted identity switches.On test, 95 swaps across 56 of 150 SportsMOT sequences contribute to the mechanism’s gain; association metrics also improve.Source: Table 5 · p. 8, §4.6 · p. 8

4.5. Long horizon: the merge and its gate

  • 84.12 HOTA on SportsMOT val follows removing the merge’s motion gate, versus 79.20 gated; the gate-free optimum lies on a robust plateau, selected by best worst-sport HOTA.The same merge configuration transfers to DanceTrack val under a different embedding model, gaining +1.87 HOTA.Source: §4.5 · p. 8

5. Conclusion

ARC-SORT uses the tracker’s uncertainty to govern association tolerance, identity commitment, and long-gap tracklet linking. Its scope is bounded by pairwise single-swap repair, offline long-horizon linking, and detector limits.

5. Conclusion

  • ARC-SORT uses IMM covariance for continuous per-track association tolerance, assignment margins to defer ambiguous identity decisions, and appearance-only linking when motion information runs out.The tracker is training-free.Source: §5 · p. 8
  • Deferred commitment is pairwise and permits only a single corrective swap, while the long-horizon re-identification stage is offline.Source: §5 · p. 8
  • Public model-zoo detections cap HOTA; DanceTrack’s 62.1 HOTA is below trained specialists, and trained or foundation-model systems retain a 0.1–5.4 HOTA edge on SportsMOT.Source: §5 · p. 8

A. Deferred commitment: fine-grained checks (full numbers)

Fine-grained checks show that deferred commitment’s validation gains vary by sport and embedding stack, while sequence-level paired deltas support gains on both test sets.

A. Deferred commitment: fine-grained checks (full numbers)

  • Basketball’s SportsMOT validation arbitration delta is +1.68, versus +0.21 for volleyball and +0.18 for football.Corrective swaps numbered 12, 11, and 2, respectively.Source: A. Deferred commitment: fine-grained… · p. 11
  • The swap audit verified 11 correct and 0 incorrect swaps among 25 SportsMOT episodes, while DanceTrack had 11 correct, 24 incorrect, and 53 undecidable among 88.The passage attributes DanceTrack’s harder pre/post attribution to crossing density; sequence-level paired deltas were +1.50 online and +1.00 test.Source: A. Deferred commitment: fine-grained… · p. 11
  • On Deep-EIoU embeddings, the validation experiment measured a +1.06 pooled delta across 144 entangled episodes and 18 swaps.Football’s failures are camera exits, which deferral leaves alone.Source: A. Deferred commitment: fine-grained… · p. 11

B. The alternatives: full definitions

The alternatives consume the same per-track prediction and covariance under identical inputs, with each mode tuned at its own matching threshold. They vary how covariance affects association.

B. The alternatives: full definitions

  • The Mahalanobis veto rejects a pair when its distance under S_m exceeds τ_M, using covariance only to gate matches.The tested thresholds were 9.49, 16, 30, 60, and 100, with fixed e=0.7; a variant stacked the gate on adaptive expansion.Source: B. The alternatives: full definitions · p. 11
  • Fixed expansion sets e_i to 0.7 for every track, while the binary velocity switch selects between two fixed factors using a normalized-speed percentile.The binary switch uses MeMoSORT-style default calibration and two more aggressive variants.Source: B. The alternatives: full definitions · p. 11
  • Anisotropic expansion sets separate per-axis factors from σ_x/w and σ_y/h to test whether directional shape helps.Source: B. The alternatives: full definitions · p. 11
  • Sigma-point expected overlap approximates expected DIoU under N(ẑ_xy, S_xy) with an unscented five-point method, optionally blended with Eq. (2).Source: B. The alternatives: full definitions · p. 11

C. Video

The video materials show uncertainty-adaptive expansion during a basketball sequence and an appearance-only merge across a football camera exit. The football episode was selected programmatically using ground-truth-verified criteria.

C. Video

  • The football video shows identity 3 preserved across an 11.7-second camera exit with appearance-only merging, while motion-gated extrapolation splits it to identity 36 at re-entry.Both runs share detections, embeddings, and camera transforms; only the merge gate differs.Source: C. Video · p. 11
  • In the basketball example, EIoU expands across a 12-frame miss and tightens on re-capture; IMM model disagreement widens S during the frame-590 burst.Constant-acceleration takes over during that burst.Source: Figure 3 · p. 11
  • The football episode was the longest ground-truth-verified gap of 293 frames meeting the programmatic selection criteria across football validation sequences.The motion-gated output had to split the gap-edge identities, and both edges had to match the same ground-truth identity.Source: C. Video · p. 11

D. Full mechanism-ladder measurements

Table 6 records the mechanism campaigns on shared cached inputs, with each campaign anchored to the same 77.97-HOTA result.

D. Full mechanism-ladder measurements

E. Full merge grid

The full merge grid compares motion-gate tolerances and gap caps, while the selected configuration favors worst-sport HOTA over the pooled peak.

E. Full merge grid

  • The grid’s pooled argmax is g450/off/0.80 at 84.19, but the selected g300/off/0.85 maximizes worst-sport HOTA.Source: E. Full merge grid · pp. 11–12
  • With the motion gate on, HOTA stays between 79.14 and 79.20 across gap caps and thresholds; removing it yields a plateau.The grid also measures a monotone tolerance response from 2 to 6 to 15 to off.Source: E. Full merge grid · p. 11
  • In the illustrated SportsMOT exit, the target re-enters after 293 frames (11.7 s); appearance-only merging preserves its identity, while the motion veto splits it 3 →36.Source: Figure 4 · p. 12
  • Table 8 provides a per-sport breakdown of the three merge configurations discussed in Section 4.5.Source: E. Full merge grid · p. 12

F. Transfer and runtime tables

These tables document cross-benchmark merge transfer and runtime, historical SportsMOT test rows, and a shared-detection comparison isolating association performance.

G. Historical baselines omitted from the SportsMOT test comparison

F. Transfer and runtime tables

H. Shared-detection study

  • With identical cached SportsMOT val detections, ARC-SORT leads seven trackers in HOTA, AssA, and IDF1 and has the fewest identity switches, despite the lowest DetA.The shared detections make the differences attributable to association alone.Source: H. Shared-detection study · p. 13, Table 11 · p. 13

I. Per-sequence results

The selected online-plus-interpolation-plus-merge configuration is reported per sequence across all 45 SportsMOT val sequences.

I. Per-sequence results

  • Per-sequence HOTA, AssA, and IDF1 are listed for all 45 SportsMOT val sequences using τ=0.85, g=300, and the motion gate off.Source: §I · p. 13, Table 12 · p. 14
Loading 2610.22306v1…