Source-linked AI summary
Can Event Detectors Leave the Broadcast Camera? Evidence from Tennis
David Soeiro-Vuong, Andre-Louis Rochet
TL;DR
The paper asks whether tennis event detectors transfer from broadcast footage to the viewpoints amateurs use, where bounces and hits can be confused. It builds a multi-viewpoint benchmark and evaluates supervised spotters, vision-language models, and a training-free arbitration layer. Detector performance varies with viewpoint: broadcast-trained spotters falter on phone and court-level clips, while RallyArb retains strong bounce scores there but has loose hit timing.
Problem
No public corpus measures whether tennis event detectors transfer from broadcast footage to the viewpoints amateur players use, despite bounces and hits being easy to confuse on monocular video.
Source: §1 · p. 1Method
Results
Across viewpoints, broadcast-trained spotters nearly fail on phone and court-level footage, while RallyArb scores 0.842 bounce F1 on held-out grass-am and vision-language models detect events on every viewpoint.
Source: §7 · pp. 8–9, §10 · p. 12Takeaways & Limitations
The benchmark measures viewpoint transfer, and the observed detector-family ranking changes with camera viewpoint and matching tolerance.
Source: §1 · p. 2, §10 · p. 12Eight clips provide only two or three clips per camera type, leaving wide bootstrap intervals and making RallyArb’s phone and court-level claim rest on one held-out clip.
Source: §8 · p. 10
Abstract
from arXiv · showhide
Ball bounces and racket hits are the events from which tennis statistics are derived, and on monocular video the two are easy to confuse. We release a benchmark of 345 bounce and hit annotations at frame level on eight clips that mix broadcast, fixed elevated, phone and court-level viewpoints, with a frozen development and test split, a confusion-aware matcher, clip-level bootstrap intervals and a blind second annotation of three clips. On it we score supervised precise spotters with published broadcast weights, two zero-shot vision-language models, and RallyArb, an arbitration layer over a ball track and player poses that fits no parameter on event labels. The supervised spotters score near-perfectly on broadcast footage and almost nothing on phone and court-level footage, where the pooled bounce F1 of E2E-Spot is 0.130. RallyArb keeps a bounce F1 above 0.8 on the three phone and court-level clips, two of which were used to tune it, and finds 8 of 11 bounces on the held-out one, but it places hits too far from the contact frame to match human timing. The vision-language models detect events on every viewpoint and miss the contact frame by about 0.1 s. Fine-tuned leave-one-clip-out from its broadcast weights, the spotter reaches about 0.8 bounce F1 on two phone and court-level clips and stays at 0.2 on a grazing phone angle. The annotations, the evaluator and the frozen predictions of every scored row are released.
1 Introduction
The paper addresses whether tennis event detectors transfer from broadcast footage to the viewpoints amateur players use. It introduces a diagnostic benchmark and evaluates detector families across those viewpoints.
1 Introduction
- Amateur players filming with a single tripod-mounted phone need the event stream—bounce locations, hit times, and their alternation—from a non-broadcast viewpoint.Existing automatic tennis analytics and ball trackers were built for broadcast settings.Source: §1 · p. 1
- Bounces and hits are both sharp discontinuities on a monocular track, so a detector tuned for one can fire on the other.A wrong-class prediction is counted as an error for both classes.Source: §1 · p. 1
- The benchmark measures transfer across viewpoints, addressing the absence of a public corpus for broadcast-to-amateur event detection.Prior precise-spotting benchmarks use broadcasts, while racket-sport benchmarks target tracking rather than bounce and hit events.Source: §1 · p. 1
- The paper evaluates published-weight supervised spotters, two vision-language models, and a training-free arbitration layer under one fixed protocol.It also separates clips each row could have been tuned on from those it could not, and measures leave-one-clip-out fine-tuning.Source: §1 · p. 2
2 Related work
Prior racket-sport analytics spans ball tracking, event annotation, and precise spotting, but much of the event and tracking work uses broadcast or fixed match footage. This paper also relates its arbitration layer to court registration and temporal localization by vision-language models.
Ball tracking in racket sports
- The arbitration layer consumes a ball tracker as an interchangeable front end, using a Kalman-filtered single-class detector in this pipeline.Prior systems include TrackNet for broadcast tennis [Huang et al., 2019], extensions to other sports and trajectory methods, and WASB [Tarashima et al., 2023].Source: §2 · p. 2
Event and stroke annotation in racket sports
- Existing racket-sport event and stroke resources cover tennis, table tennis, and badminton, but use broadcast or fixed match footage.They include dense event labels, rally segmentation, stroke labels, and tools to reduce annotation effort.Source: §2 · p. 2
Precise event spotting
- Precise-spotting work includes broadcast tennis datasets and frame-level spotters, while RacketVision benchmarks ball and racket analysis without bounce and hit events.The E2E-Spot tennis dataset extends Vid2Player to 28 US Open and Wimbledon broadcast matches [Hong et al., 2022].Source: §2 · p. 2
Temporal localization by vision-language models
- Vision-language models can answer when an event happens, but temporal predictions may change under small query changes [Jung et al., 2025].The paper tests whether a frame-indexed prompt can place ball contact within a few frames.Source: §2 · p. 2
Court registration and structured decoding
- Unlike broadcast court-calibration approaches, the arbitration layer needs no court registration and decodes trajectory cues with a Viterbi-style dynamic program.It uses Theil–Sen slopes to read trajectory directions.Source: §2 · pp. 2–3
3 Problem statement
The task is to label bounce and racket-contact events in monocular video, matching each untagged event to a prediction of its class within a frame-rate-dependent tolerance. A joint matcher treats wrong-class predictions as errors for both classes.
3 Problem statement
- Each untagged bounce or racket-contact event must match a prediction of the same class within τ = round(0.15 ν) frames.Ground-truth events are represented by their first contact frame, with tags that determine which events are excluded from scoring.Source: §3 · p. 3
- A greedy one-to-one matcher pairs both classes jointly, nearest frame first; a wrong-class pair counts as a false positive and false negative.Excluded ground-truth events are removed after matching, so predictions matched to them are ignored.Source: §3 · p. 3
- The 0.15 s tolerance is wider than the 1–2 frames used by precise-spotting benchmarks, and results are also reported from 0 to 0.5 s.Wrong-class discontinuities lose both precision and recall; misses without confusion lose recall only.Source: §3 · p. 3
4 The benchmark
The benchmark combines eight clips spanning multiple surfaces, frame rates, and camera viewpoints, with versioned annotations, a frozen split, and a common evaluator. It also reports annotation agreement and clip-level uncertainty.
4 The benchmark
- Eight clips span clay, grass, and hard courts at 25–60 frames per second, covering broadcast, elevated, phone, and court-level viewpoints.The corpus table groups clips as broadcast, fixed elevated, and phone or court level.Source: §4 · p. 3, Table 1 · p. 4
- Every physical contact is annotated at its first contact frame, with tags used to exclude events such as serves, tosses, far events, and replays from scoring.The policy is versioned, and the far half is scored only where it can be verified.Source: §4 · pp. 3, 5
- The synthetic viewpoint illustration shows far bounces and hits only a few pixels apart near the horizon in the two low-camera views.It contains no benchmark data; the companion figure shows the viewpoints on real frames.Source: Figure 1 · p. 4, Figure 2 · p. 5
- A blind second annotation agreed on all 130 matched labels with no bounce-hit confusions; rally-only bounce and hit F1 were 1.000 and 0.957.Across matched pairs, frames differed by 0.24 on average, and 79% coincided exactly.Source: §4 · p. 5
- Bootstrap intervals resample clips rather than events because events within a clip are correlated; with four clips per split, intervals are wide.The intervals are reported to prevent over-reading pooled differences and do not support significance claims.Source: §4 · p. 6
- The frozen development split permits tuning, while the test split was scored once per system version and its pipeline outputs were not inspected during development.The test events were annotated before tuning began.Source: §4 · p. 6
- Replay fixtures provide tracked ball centres, validity masks, and identity-locked player poses, enabling evaluation of downstream methods without source videos.Methods that read pixels still require the videos.Source: §4 · p. 6
5 Detector families
The section evaluates supervised spotters, zero-shot vision-language models, and RallyArb under fixed protocols. It also tests whether leave-one-clip-out fine-tuning can teach a broadcast spotter to handle the corpus’s other viewpoints.
Supervised precise spotters
- E2E-Spot and T-DEED use released tennis weights without retraining, with fixed input preprocessing, class mapping, thresholds, and non-maximum suppression.The selected T-DEED checkpoint was designated beforehand.Source: §5 · p. 6
Vision-language models
- Gemini 3.8 Flash and Qwen3-VL-235B-A22B receive sampled frame windows and a fixed prompt defining each event.The models and sampling rate were chosen on development data; the prompt was not edited after results were read.Source: §5 · p. 6
RallyArb
- RallyArb combines five unlabeled candidate generators with trajectory and wrist-distance evidence, rule-based hypothesis removal, and a dynamic program enforcing rally alternation.It runs downstream of unchanged ball-tracking and pose-estimation models and fits no parameter on event labels.Source: §5 · p. 6
- RallyArb’s constants are ratios of frame rate, frame size, or player-box height, fixed on demo3 and refined on four development clips.The paper specifies that rallies alternate hits and bounces at a beat estimated per clip.Source: §5 · p. 7
Fine-tuned spotter
- Leave-one-clip-out fine-tuning trains the smaller E2E-Spot weights on the other seven clips to measure learnability, not to produce a deployable detector.The published head is retained, and settings were fixed before the first fold; each fold ran once.Source: §5 · p. 7
6 Evaluation protocol
All methods use a common evaluator and rally-only scoring view, but test-set pooling can favor broadcast-strong methods. The protocol also records what information each method could access.
Scoring and test composition
- 72 of 140 rally-only test events come from two broadcast clips, so per-camera results are the primary reading because pooled scores favor broadcast-strong methods.The event order fixes what each row could have seen, and Table 2 lists this information per row.Source: §6 · p. 7
- The common evaluator uses tolerance round(0.15 ν) and excludes serve, toss, far, dead, offscreen, and replay tags from rally-only scoring.Including dead bounces changes test bounce F1 from 0.621 to 0.634 for RallyArb and from 0.776 to 0.750 for E2E-Spot.Source: §6 · p. 7, Table 2 · p. 7
Information regimes
- Table 2 records each row’s information access: development clips are out of sample except for RallyArb and Gemini, while fine-tuned test folds train on the other three test clips’ labels.The table also lists diagnostic executions that were not used for any choice.Source: Table 2 · p. 7, §6 · p. 7
7 Results
Published broadcast-trained spotters perform near-perfectly on broadcast clips but poorly on phone and court-level footage; RallyArb retains strong bounce scores there, while fine-tuning improves spotter performance except on a grazing phone angle. Vision-language models detect events across viewpoints, but their timing is less precise than the spotters’.
Result 1
- 0.989 bounce F1 and 1.000 hit F1 on broadcast clips contrast with 0.130 and 0.154 on phone and court-level clips for E2E-Spot.The camera-type comparison is shown in Table 3.Source: §7 · p. 8, Table 3 · p. 8
Result 2
- 0.415–0.525 bounce F1 for Gemini across every camera type shows viewpoint-spanning detection, while test results include 59 spurious predictions and 9 confusions.Qwen3-VL has 256 spurious test predictions and pooled test F1 of 0.190 and 0.257.Source: §7 · pp. 8–9
Result 3
- 0.828 bounce F1 on felix and hard-pro and 0.842 on grass-am make RallyArb stronger than other rows on phone and court-level clips, but two clips informed its constants.On held-out grass-am, it scores 0.842 bounce F1 and 0.556 hit F1; Gemini leads on hits.Source: Table 3 · p. 8, §7 · p. 9
Result 4
- 0.776 and 0.905 pooled test F1 put E2E-Spot ahead of RallyArb, while RallyArb leads on development at 0.852 and 0.646; split composition affects the pooled ranking.Two broadcast clips contribute 72 of the test split’s 140 events; per-camera readings are less dependent on that composition.Source: Table 4 · p. 8, §7 · p. 9
Result 5
- 0.846 and 0.941 pooled test F1 for fine-tuned E2E-Spot improve on phone and court-level clips, but felix remains at 0.200 and 0.261.These leave-one-clip-out results are cross-validation scores, with each test fold trained on other test clips.Source: Figure 3 · p. 9, §7 · p. 10
Result 6
- 0.009 and 0.019 s average contact-frame offsets for E2E-Spot are smaller than RallyArb’s 0.105 s hit offset and the vision-language models’ offsets.RallyArb anchors hits on wrist-speed peaks rather than the contact frame; Figure 3 compares bounce and hit F1 by camera type and includes fine-tuned E2E-Spot.Source: Figure 3 · p. 9, §7 · p. 10
8 Threats to validity
The benchmark’s small, partly singly annotated corpus and several evaluation choices constrain how broadly its comparisons can be interpreted. The reported viewpoint effect is consistent across the phone and court-level clips, but fine-grained differences remain uncertain.
8 Threats to validity
- Five of the eight clips have a single annotation, and the test tag policy was informed by an audit of RallyArb’s development-set spurious predictions.No test event was retimed; the two annotators disagreed on two development events.Source: §8 · p. 10
- With only eight clips, test intervals reach 0.61 in width, and the RallyArb group claim rests on one held-out phone or court-level clip.The viewpoint effect is consistent across the three phone and court-level clips, but smaller differences can turn on one or two events.Source: §8 · p. 10
- Published spotters’ broadcast scores may benefit from training on 28 US Open and Wimbledon broadcasts, which overlap in kind with two professional clips.Clay-pro lies outside that distribution but scores as highly as the other broadcast clips.Source: §8 · pp. 10–11
- Fine-tuned test folds use labels from the other three test clips, and the reported results apply only to the smaller RegNetY-002 model and these settings.The spotters’ exact-frame scores are also bounded by running them on every second frame of 50 and 60 fps clips.Source: §8 · p. 11, Figure 4 · p. 11
- For phone and court-level clips, the ball falls outside the spotters’ center crop at 15 of 32 tracked rally events on felix, 6 of 21 on hard-pro, and 3 of 18 on grass-am.The vision-language models were tested with one prompt and sampling configuration each, set on the development split.Source: §8 · p. 11
- RallyArb’s scores measure the arbitration layer together with its ball tracker and pose estimator, and would change with a better tracker.The source videos are not redistributed, though replay fixtures allow downstream-of-tracking methods to be scored without them.Source: §8 · p. 12
9 Reproducibility
The release provides the annotations, evaluator, frozen predictions, and materials needed to reproduce the reported scores and rebuild the paper’s figures. The source videos are not redistributed, but replay fixtures support scoring methods downstream of tracking.
9 Reproducibility
- The repository releases eight ground-truth sidecars, the annotation policy, matcher, multi-clip evaluator, replay fixtures, and frozen predictions with row manifests.It also includes evaluator reports and the double-annotation protocol and results.Source: §9 · p. 12
- The standard-library evaluator reproduced every pooled number in Tables 3 and 4 exactly from the released files on 2026-10-26.The evaluator requires Python 3.9 or later and no other package; scripts rebuild the tables and figures.Source: §9 · p. 12
- Source videos are not redistributed, but replay fixtures permit scoring methods that work downstream of tracking without the videos.The arbitration-layer code is available from the authors on request.Source: §9 · p. 12
10 Conclusion
On this small mixed-viewpoint benchmark, detector rankings follow the camera: broadcast-weight spotters are strong on broadcasts but nearly silent on phone and court-level footage. RallyArb maintains strong bounce scores on those latter clips, while its hit timing is loose; fine-tuning helps except at a grazing angle.
10 Conclusion
- Broadcast-weight spotters are near-perfect on broadcast footage and nearly silent on phone and court-level footage, while vision-language models detect across viewpoints but miss contact by about 0.1 s.RallyArb exceeds 0.8 bounce F1 on the three phone and court-level clips, including the held-out clip, but places hits too loosely; fine-tuning reaches those viewpoints except the grazing one.Source: §10 · p. 12
A Per-clip scores
- Table 5 reports rally-only bounce F1 and hit F1 per clip and row; RallyArb’s constants were fixed on demo3 and refined on the four development clips.Each test value is a single execution.Source: Table 5 · p. 14
B Additional released rows
- The additional released rows include a camera-type routing rule written after both component rows were scored on test, and configurations combining zero-shot bounce frames with RallyArb.The paper explicitly draws no conclusion from these further rows.Source: B Additional released rows · pp. 14–15
C The arbitration layer
RallyArb combines candidate generation, trajectory and contact evidence, label-legality rules, and sequence decoding without fitting parameters on event labels. Its configuration uses real detections for directional evidence and enforces rally alternation through a beat-based dynamic program.
C The arbitration layer
- The test configuration reads a ball track with a per-frame validity mask and two identity-locked players’ boxes and 17 keypoints; unreadable measurements abstain.A spline-completed track is used for lookup only; directional evidence is read from real detections.Source: C The arbitration layer · p. 15
- Bounce candidates pass line-fit, slope, residual, restitution, and energy-ratio tests, while hit candidates are wrist-speed peaks gated by width and side-scaled amplitude.Three trajectory generators also propose image-height extrema, sharp velocity turns, and near-court knees; nearby candidates are clustered.Source: C The arbitration layer · p. 15
- Robust slopes estimate motion on either side of a cluster; a descending-to-ascending vertical flip with preserved horizontal direction supplies floor evidence.Contact distance is normalized by player-box height; on the development rally, real contacts were at or below 0.51 and phantom swing anchors at or above 0.90.Source: C The arbitration layer · p. 15
- Label-legality rules exclude bounce labels for hit-like clusters and hit labels for clear floor evidence without a swing or redirect, before scoring can restore them.Clusters with floor evidence or a corroborated swing at the wrist become mandatory bounce or hit anchors, respectively.Source: C The arbitration layer · p. 15
- A dynamic program emits or skips clusters while enforcing class alternation according to beat counts between anchors and preventing immediate repeats; mandatory anchors cannot be skipped.The beat is estimated from median anchor gaps, and the decoder uses a 256-state beam.Source: C The arbitration layer · p. 15