Causal neural set filtering for online multi-target tracking

1. INTRODUCTION

• Classical Bayesian trackers explicitly model recursive state, association, and existence under clutter, missed detections, uncertain associations, and target birth and death, but complexity may grow with association uncertainty.Source: §1 · p. 1
• Track-MT3 propagates learned query information but repeatedly re-encodes overlapping measurement windows, and its queries do not explicitly carry kinematic uncertainty or target existence.MT3 and Track-MT3 formulate point-measurement tracking as Transformer-based set prediction with temporal context recovered from measurement windows.Source: §1 · p. 1
• CNSF encodes only current measurements and carries past evidence in a structured recursive track state, using exclusive Sinkhorn association, Kalman-shaped updates with moment matching, and recurrent Bernoulli lifecycle modeling with measurement-driven birth.The mechanisms support recursion under measurement competition, association ambiguity, missed detections, and birth–death transitions.Source: §1 · p. 1
• 19.3% lower mean GOSPA and 30.4% lower T-GOSPA than Track-MT3, with 55.9% fewer parameters and a 3.76× single-thread CPU speedup.CNSF also achieves the lowest mean GOSPA and T-GOSPA among evaluated methods on the held-out three-regime simulated test set.Source: §1 · p. 1

2. RELATED WORK

• Classical MTT uses recursive Bayesian estimation; JPDA/MHT address association ambiguity, while δ-GLMB and PMBM model existence, birth, and death.Association marginalization or hypothesis management can become costly as ambiguity grows.Source: §2 · p. 1
• CNSF integrates soft exclusive association and association-conditioned moment matching with recurrent existence modeling and measurement-driven birth.Related approaches include learned Sinkhorn–Kalman coupling [17].Source: §2 · p. 1
• MT3 and Track-MT3 extend learned set prediction to point-measurement MTT, while CNSF propagates a structured recursive track state and encodes only current measurements.Related persistent-query designs include TrackFormer, MOTR, GTR, TransMOT, and MOTRv2.Source: §2 · p. 1

3.1. Recursive state and neural–physical prediction

• CNSF carries temporal information in the recursive track state, encoding current measurements once rather than re-encoding historical measurements.The updated track state is the only persistent state passed to the next step.Source: §3 · p. 1, §3.1 · p. 2

3.2. Exclusive association and uncertainty propagation

• Sinkhorn association imposes soft one-to-one constraints, while Kalman-shaped PAIR updates and moment matching preserve association-induced covariance.PAIR and MISS hypotheses collapse into one recursive state per track rather than propagating multiple association hypotheses.Source: §3.2 · p. 2

3.3. Bernoulli lifecycle and network architecture

• Recurrent lifecycle memory and bounded existence-log-odds corrections use association, objectness, ambiguity, time, covariance, and age cues; birth heads initialize new-track states.Updated existence controls track confirmation, retention, and termination.Source: §3.3 · p. 3

3.4. Training objective

• Training preserves recursive identities through cross-frame target alignment, using Hungarian assignment only for unmatched birth candidates and averaging losses across valid rollout frames.The objective includes set, risk, association, birth, lifecycle, survival, and cardinality terms; training also uses closed-loop curricula and joint BPTT through the recursive path.Source: Figure 1 · p. 2, §3.4 · p. 3

4.1. Protocol and training configuration

• The held-out test covers 50 trajectories in each of three 100-frame regimes; frames 20–99 are scored using GOSPA, Pro-GOSPA, and T-GOSPA.The regimes increase in difficulty, with varying birth, survival, detection, noise, and clutter settings.Source: §4.1 · p. 3

4.2. Results and controlled ablations

• CNSF achieves the lowest mean GOSPA and T-GOSPA; versus Track-MT3, the metrics fall 19.3% and 30.4%, while latency drops from 117.37 to 31.18 ms/frame with 55.9% fewer parameters.Table 1 reports the held-out test comparison.Source: Table 1 · p. 4, §4.2 · p. 4

4.1. Protocol and training configuration

• Training uses on-the-fly trajectories, AdamW for 20k updates, and a BPTT horizon that grows from 20 to 100 frames, while inference remains one-frame recursive.Physical difficulty reaches its full regime by step 5k, association is fully predicted by step 4k, and the BPTT horizon reaches 100 frames by step 14k.Source: §4.1 · p. 4

4.2. Results and controlled ablations

• At comparable capacity, CNSF records G/T = .882/1.004 versus 1.148/1.595 for Track-MT3-CM, and latency falls 21.4% from 39.68 to 31.18 ms/frame.Track-MT3-CM has 8.48M parameters and CNSF 8.54M.Source: §4.1 · p. 3, §4.2 · p. 4
• In dense S3, CNSF and Track-MT3 have similar frame-wise GOSPA (1.6518 vs. 1.6803), but CNSF has lower localization error and T-GOSPA (1.8837 vs. 2.218).CNSF has slightly higher S3 missed/false penalties, so its S3 GOSPA gain mainly reflects improved localization rather than cardinality handling.Source: §4.2 · p. 4
• Removing exclusive Sinkhorn causes the largest mean T-GOSPA increase (1.004→1.659) and raises S3 false error from 0.246 to 1.325.The lifecycle replacement raises mean T-GOSPA to 1.248 and S3 missed error to 1.195 while lowering false error to 0.177; replacing pairwise mixing increases all three mean metrics, with a smaller T-GOSPA rise (1.004→1.060).Source: §4.2 · p. 4, Table 2 · p. 4

• CNSF encodes only current measurements and propagates history through a structured recursive track state.Source: §5 · p. 4
• On the held-out test set, CNSF achieves the lowest mean GOSPA and T-GOSPA among evaluated methods and outperforms capacity-matched Track-MT3-CM in accuracy and speed.Source: §5 · p. 4
• CNSF’s main limitation is compressing prolonged association ambiguity into a single moment-matched state.Source: §5 · p. 4