Source-linked AI summary
SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos
Anthony Cioppa, Silvio Giancola, Adrien Deliege, Le Kang, Xin Zhou, Zhiyu Cheng, Bernard Ghanem, Marc Van Droogenbroeck
TL;DR
Player tracking can support player and team statistics, but soccer MOT datasets and benchmarks have been limited. SoccerNet-Tracking provides annotated short sequences and a full half-time video for benchmarking. The benchmark highlights that current methods still struggle with fast movement and heavy occlusion.
Problem
Soccer MOT datasets and benchmarks for training and evaluating methods on a common testbed have been limited, despite tracking’s relevance to player and team statistics.
Source: §1 · pp. 1–2Method
The paper introduces SoccerNet-Tracking, with annotated 30-second soccer sequences and a complete 45-minute half-time video, and benchmarks state-of-the-art MOT methods.
Source: §1 · p. 2Results
Current trackers do not yet handle soccer cases involving high-speed movement and highly occluded objects correctly.
Source: §5 · p. 8Takeaways & Limitations
The dataset establishes a first benchmark intended to support better soccer tracking, including long-term re-identification in challenging environments.
Source: §5 · p. 8Detection remains incomplete for fast-moving players, and current trackers lack capabilities for hard cases.
Source: §4 · p. 8
Abstract
from arXiv · showhide
Tracking objects in soccer videos is extremely important to gather both player and team statistics, whether it is to estimate the total distance run, the ball possession or the team formation. Video processing can help automating the extraction of those information, without the need of any invasive sensor, hence applicable to any team on any stadium. Yet, the availability of datasets to train learnable models and benchmarks to evaluate methods on a common testbed is very limited. In this work, we propose a novel dataset for multiple object tracking composed of 200 sequences of 30s each, representative of challenging soccer scenarios, and a complete 45-minutes half-time for long-term tracking. The dataset is fully annotated with bounding boxes and tracklet IDs, enabling the training of MOT baselines in the soccer domain and a full benchmarking of those methods on our segregated challenge sets. Our analysis shows that multiple player, referee and ball tracking in soccer videos is far from being solved, with several improvement required in case of fast motion or in scenarios of severe occlusion.
1. Introduction
SoccerNet-Tracking introduces a soccer multiple-object-tracking dataset and benchmark to address the limited availability of soccer tracking data. The paper motivates tracking as a source of player and team statistics while highlighting the sport’s difficult visual conditions.
1. Introduction
- Player tracking supplies positions over time needed to estimate statistics such as endurance, distance run, speed, possession, assists, and goals.Video-based tracking can gather these statistics without invasive sensors placed on players.Source: §1 · p. 1
- Similar player appearances, mutual occlusions, and the ball’s small size, occlusions, fast motion, blur, and changing shape make soccer tracking challenging.The paper also identifies trajectories as useful for action spotting and highlight generation.Source: §1 · p. 2
- SoccerNet-Tracking provides 200 30-second sequences and an annotated 45-minute half-time video for tracking players, balls, and referees.The sequences cover challenging soccer scenarios, and the half-time video supports long-term tracking.Source: §1 · p. 2
2. Related Work
The related work spans general multiple-object tracking and sports-video analysis, while identifying soccer player tracking as a domain with limited public multi-object tracking data. SoccerNet-Tracking is positioned as a complement to existing sports benchmarks and prior player-localization and tracking work.
2. Related Work
- The paper frames its contribution within research on multiple-object tracking and its applications to sports analysis.The related work covers both datasets and methods for these areas.Source: §2 · p. 2
2.1. Multiple Object Tracking (MOT)
- Tracking-by-detection localizes objects in each frame and links detections into tracklets; methods address association, disappearance, and detector failure.The paper discusses SORT, DeepSORT, Tracktor, FairMOT, and ByteTrack as examples of MOT approaches.Source: §2.1 · p. 2
2.2. Sports Video Understanding
- SoccerNet’s earlier benchmarks cover action spotting, replay grounding, player re-identification, and pitch localization; this work adds multiple-object tracking.The proposed soccer data includes 200 tracking sequences and a fully annotated 45-minute half-time video.Source: §2.2 · p. 3
2.3. Player Localization and Tracking in Sports.
- Prior soccer player tracking often used small or private datasets; unlike SSET’s short single-object sequences, this work provides longer multi-object clips and public long-term tracking data.Figure 2 compares datasets by bounding boxes, frames, and unique tracklets; its area represents unique tracklets.Source: §2.3 · p. 3, Figure 2 · p. 3
3. SoccerNet-Tracking Dataset
SoccerNet-Tracking combines challenging soccer clips with a complete annotated half-time for long-term tracking. Its dataset is large by tracking benchmarks’ sequence and bounding-box counts, and includes persistent identities and metadata for several object classes.
3. SoccerNet-Tracking Dataset
- The dataset uses main-camera footage from 12 complete 2019 Swiss Super League games, recorded at 1080p and 25 frames per second.It includes 200 non-overlapping 30-second clips selected around challenging action spots, plus a complete half-time chosen for challenging actions and diversity.Source: §3 · pp. 3–4
- SoccerNet-Tracking has the largest number of bounding boxes and sequences, and the second most frames and unique tracklets among compared tracking datasets.It is the first multi-object tracking dataset in soccer, scaling up from the earlier SSET single-object tracking dataset.Source: Table 1 · p. 4, §3 · p. 5
- The selected clips center on challenging actions, including corners and cards, where clustered players and occlusions complicate tracking.The 12 games contain no red-card or yellow-to-red-card events.Source: §3 · p. 4, Figure 3 · p. 4
- Annotations cover players, goalkeepers, referees, balls, and other objects, with team tags and visible jersey numbers for players and goalkeepers.Around 3.6 million bounding boxes cover 5,009 unique objects; 96% of players and goalkeepers have an assigned jersey number, and 11.5% of boxes are interpolated.Source: §3 · pp. 4–5
- Objects retain the same ID when they leave and re-enter the frame during a clip, supporting long-term re-identification in a setting where current trackers often lack it.The authors frame this identity policy as closer to real-world soccer performance analysis.Source: §3 · p. 5
- The games are divided into train, test, and two challenge sets; the latter include the complete half-time in the second challenge.The data structure follows MOT20 conventions, while challenge ground truth remains private and is evaluated through a submission platform.Source: §3 · pp. 5–6
4. Benchmarks
The benchmark compares tracking methods with and without ground-truth detections and finds that fine-tuning FairMOT improves detection and association. Results also expose distinct difficulties in event classes, especially fast motion and dense occlusion.
Task and Metrics
- The benchmark evaluates ground-truth-detection re-identification and raw-video tracking, using HOTA as its main metric alongside MOTA.With ground-truth detections, the setup focuses on association performance.Source: §4 · p. 6
Baselines
- DeepSORT adds appearance features to association, FairMOT jointly detects and generates re-identification features, and ByteTrack associates every detection box.ByteTrack uses IoU similarities for low-score detections.Source: §4 · p. 6
Main Results
- Fine-tuned FairMOT improves detection and association, achieving state-of-the-art HOTA and MOTA; without ground-truth detections, performance rises from DeepSORT to FairMOT to ByteTrack.Table 3 presents the leaderboard across both detection setups.Source: §4 · p. 7, Table 3 · p. 7
Per-class Analysis
- Class difficulty shifts by setup: corners challenge association with ground-truth detections, while goals and penalties challenge detection without them; substitutions and yellow cards are easiest.The per-class comparison reports HOTA (DetA/AssA) for ByteTrack in both setups.Source: §4 · p. 7, Table 4 · p. 7
Qualitative Results
- ByteTrack tracks players through partial occlusions, but fast motion and densely clustered free-kick players produce challenging associations and poor tracking.The authors call for improved re-identification to handle such hard cases.Source: §4 · p. 8, Figure 5 · p. 8
- Fine-tuning FairMOT excludes audience detections and detects some fast-moving players, although motion blur remains challenging.The qualitative comparison shows predictions before and after fine-tuning.Source: §4 · p. 8, Figure 6 · p. 8
5. Conclusion
SoccerNet-Tracking establishes a soccer MOT benchmark and examines what current methods still struggle to track. The authors present it as a starting point for improving tracking, including long-term re-identification in challenging environments.
5. Conclusion
- SoccerNet-Tracking is presented as the largest soccer multi-object tracking dataset, featuring high-speed movement and highly occluded objects.The authors study three state-of-the-art tracking methods and discuss challenging situations they do not correctly handle.Source: §5 · p. 8
- The authors set a first benchmark and aim to encourage better tracking methods, including long-term re-identification in challenging environments.They also aim to organize tracking challenges.Source: §5 · p. 8
- The authors describe tracking as important to research and industrial applications with potential direct impact on soccer.Source: §5 · p. 8