Source-linked AI summary
A Top-Down Framework for Metric-Scale Athlete Localization from Single Broadcast Frames
Thanh-Khoi Nguyen, Hoang-Phuc Nguyen, Linh-Huynh, Minh-Triet Tran
TL;DR
Single-frame metric-scale athlete localization is difficult because high-resolution broadcast footage combines extreme scale variance with perspective distortion. The paper proposes adaptive semantic tiling, a two-keypoint RTMPose-X estimator, and calibrated ray casting, achieving strong public-test performance and private-set generalization.
Problem
Single-frame broadcast localization must recover metric athlete positions despite extreme scale variance and perspective distortion.
Method
The method combines Boundary-Aware Adaptive Tiling with a geometrically coupled pelvis–ground-projection estimator and camera-calibrated ray casting.
Results
97.44 LocSim and 0.9128 AP are achieved on the public test set, with performance generalizing to the private set.
Takeaways & Limitations
The framework mitigates boundary-splitting under extreme scale variance and supports robust metric-scale localization across held-out arenas.
Takeaways & Limitations
The sequential pipeline incurs latency from per-instance pose inference, motivating batched inference, lightweight models, distillation, and hybrid designs.
Abstract
from arXiv · showhide
Accurate world-coordinate localization of athletes from single-frame broadcast footage is inherently challenging due to extreme scale disparities in ultra-high-resolution imagery. In this paper, we propose a top-down framework for metric-scale athlete localization from a single calibrated frame. Our approach centers on three key contributions. First, we propose Boundary-Aware Adaptive Tiling, a semantics-guided extension of standard sliced inference. By iteratively expanding tile boundaries based on coarse bounding-box predictions, it systematically ensures full object containment, effectively mitigating boundary-splitting artifacts through a lightweight pipeline adaptation without architectural modifications. By substantially mitigating recall degradation under extreme scale variance, Boundary-Aware Adaptive Tiling enables us to isolate perspective distortion as the primary source of residual localization error. Second, we adapt the RTMPose-X architecture into a specialized two-keypoint estimator (pelvis and ground projection), employing a reformulated Gated Attention Unit optimized for this geometrically coupled point pair, and then deterministically lift the 2D ground projections into world coordinates via camera-calibrated ray casting. On the public test set, our method achieves a LocSim score of 97.44 and an mAP of 0.9128, outperforming the baseline by over 21 \% and establishing a robust solution for high-resolution scale variance.
1. INTRODUCTION
The paper targets single-frame metric-scale athlete localization under extreme scale variation and perspective distortion. It combines boundary-aware detection with geometrically coupled two-keypoint estimation to improve world-coordinate accuracy.
- Motivation: Single-frame localization must detect athletes across a 4K half-pitch image and estimate metric coordinates for their pelvis ground projections.The benchmark evaluates localization in world space using calibrated camera imagery.
- Challenges: Extreme scale variance makes distant athletes occupy only a handful of pixels, causing downsampled full-frame detectors to miss small targets.Perspective distortion further requires identifying the pelvis rather than relying on coarse image-level localization.
- Approach: The proposed two-stage pipeline uses coarse detections to guide tiles, then estimates two geometrically coupled points for each athlete.The two points are the pelvis and its ground projection.
- Results: 97.67 LocSim on the private test set significantly outperforms the organizer baseline of 77.3.A ground-truth-box ablation leaves only an approximately 1.4% LocSim gap, identifying pose estimation as the primary remaining bottleneck.
- Contributions: Boundary-Aware Adaptive Tiling expands tile boundaries around partially contained detections, ensuring full object containment without additional annotations.This addresses boundary-splitting artifacts in sliced inference without architectural modifications.
2. RELATED WORK
Related work spans small-object detection, top-down pose estimation, and camera-geometry-based localization. The paper positions BAAT and a specialized two-keypoint top-down model as responses to limitations in these approaches.
- Small-Object Detection: Regular-grid sliced inference improves small-object recall but introduces boundary-splitting artifacts that semantic adaptive methods aim to eliminate.BAAT uses coarse detections to expand tile boundaries and guarantee object containment.
- Pose Estimation: Full-body pose architectures over-parameterize SynLoc because the task evaluates only the pelvis and ground-projection keypoints in world space.The paper therefore contracts the output space to the geometrically coupled point pair.
- Pose Estimation: The SynLoc baseline uses a bottom-up two-keypoint formulation, whereas the proposed method integrates top-down detection with adapted RTMPose-X.The paper states that the baseline degrades detection recall and projection accuracy relative to alternative paradigms.
- Geometric Localization: Prior camera-geometry approaches include calibration pipelines, field-line correspondence optimization, and holistic video reconstruction.These approaches predominantly rely on temporal context or continuous tracking, unlike the single-frame setting addressed here.
3. METHOD
The method is a two-stage top-down pipeline: adaptive high-resolution detection followed by specialized two-point pose estimation and calibrated world-coordinate projection.
- Pipeline: The pipeline detects athletes with adaptive tiling, estimates pelvis and ground-projection keypoints, and projects the latter into world coordinates.The projection uses deterministic ray casting against the pitch plane and camera calibration.
- Player Detection via Adaptive Tiling: BAAT begins with coarse YOLO detections on a downsampled 4K frame and initializes a SAHI grid with 1280 × 1280 slices and 0.2 overlap.The coarse detections provide semantic guidance for high-resolution crops.
- Player Detection via Adaptive Tiling: BAAT iteratively expands each window around intersecting but incompletely contained coarse boxes using a 5-pixel safety margin.Expansion is capped at 100 iterations and empirically converges within three to five iterations.
- Two-Point Keypoint Estimation: RTMPose-X is contracted from 17 output keypoints to two targets: the pelvis and its orthogonal ground projection.This removes full-body joints that are irrelevant to metric localization.
- Two-Point Keypoint Estimation: The contracted GAU forms a dense 2 × 2 attention matrix that models the physical relationship between the pelvis and ground projection.Their relative image displacement encodes height under foreshortening, encouraging implicit camera-projection modeling.
4. EXPERIMENTS
The benchmark evaluates generalization using synthetic-athlete 4K broadcast frames across multiple arenas, with held-out public and private test arenas.
- Dataset: The dataset contains 4K broadcast frames with synthetic athletes composited onto real-world backgrounds across 17 arenas.It provides image-space keypoints, 3D metric ground truths, and explicit camera calibration parameters.
- Dataset: The public test contains 9.3K images and the private challenge set 11.4K images, both using arenas held out from 42.5K training images.This protocol tests cross-arena generalization.
Dataset.
The benchmark evaluates metric-scale athlete localization using a world-coordinate similarity metric, comparing the proposed framework with a bottom-up two-keypoint baseline across public and challenge sets. The reported experiments also detail the detection and pose-estimation configurations used.
- Metric and Baseline: mAP-LocSim evaluates world-coordinate precision by replacing bounding-box IoU with metric-scale localization similarity.The similarity uses ground-plane Euclidean distance in meters, with τ = 1m.
- Main Results: Table 1 compares the organizer baseline and proposed method on public-test and challenge sets, with AP unavailable for the challenge set.Green entries indicate improvements over the baseline.
- Metric and Baseline: The organizer baseline is a bottom-up YOLOX-pose framework that regresses the same two keypoints and achieves a public-test LocSim of 76.17.Standard 17-keypoint estimators are geometrically incompatible with deterministic ground-plane ray casting because foot joints may not lie on Z = 0 during dynamic actions.
- Experimental Configuration: BAAT uses a coarse full-frame detector pass to guide adaptive 1280 × 1280 tiles, while the pose model is trained as a specialized localization stage.The implementation uses 0.2 tile overlap and a 0.001 coarse-pass confidence threshold; pose training uses adapted RTMPose-X with AdamW over 700 epochs.
- Main Results: 97.44 LocSim@t = 1 and 0.9128 AP@0.50:0.95 surpass the organizer baseline by 21.27 and 21.70 points, respectively, on the public test set.The framework also attains a private challenge-set LocSim of 96.67 versus the baseline’s 77.30.
5. ABLATION STUDY
The ablations show that Boundary-Aware Adaptive Tiling improves small-object detection by recovering missed athletes and eliminating boundary-splitting artifacts, while the two-keypoint formulation closely follows perspective geometry. Ground-truth detection adds little to final localization, identifying pose estimation as the main remaining bottleneck.
- Effect of Tiling Strategy on Detection: Approach A1 combines SAHI-trained weights with Boundary-Aware Adaptive Tiling to eliminate boundary-splitting artifacts and achieve the best performance across nearly all detection metrics.Approach B2 marginally leads only on APL.
- Effect of Tiling Strategy on Detection: Approach C records APS of 0.7668, whereas fixed SAHI inference in Approach B2 raises APS to 0.8220 by recovering missed small instances.The comparison isolates the benefit of tiled inference before adaptive boundary handling.
- Effect of Tiling Strategy on Detection: Qualitative outputs show progressively recovered far-touchline athletes from Approach C through B2 to A1, with boundary-splitting artifacts eliminated under A1.The comparison concerns full-image inference, fixed SAHI, and Boundary-Aware Adaptive Tiling.
- Implicit Geometric Learning in the Two-Keypoint Formulation: Predicted pelvis-to-ground displacement achieves a curve-level MAE of 0.333px and Pearson correlation of 0.8963 against the reference perspective curve.The curves are compared using distance-binned medians across camera-to-athlete distances.
- Detection Upper Bound: Replacing predicted boxes with ground-truth boxes raises LocSim from 97.44 to 98.86, a marginal 1.42-point gain that places the main residual error in pose estimation.The upper-bound analysis holds the pose estimator and ray-casting stages fixed.
6. CONCLUSION
The paper presents a two-stage top-down framework combining adaptive tiling with two-keypoint pose estimation and ray casting for single-frame metric-scale athlete localization. It reports strong benchmark performance while identifying efficiency and geometric modeling as future directions.
- 6. CONCLUSION: The framework combines Boundary-Aware Adaptive Tiling with a two-keypoint RTMPose-X formulation and ray casting to recover metric-scale athlete coordinates from broadcast imagery.The two keypoints are pelvis and ground projection.
- 6. CONCLUSION: The method achieves 97.44 LocSim and 0.9128 AP, outperforming the baseline by over 21 points and generalizing to the private set.These are the paper’s reported headline outcomes.
- 6. CONCLUSION: Future work targets efficiency and geometric modeling, including batched inference, lightweight models, distillation, hybrid designs, and geometry-aware supervision.The sequential pipeline incurs latency due to per-instance pose inference.