Prompt, Probe, Train, or Annotate? Single-camera sports video understanding in amateur settings

1 Introduction

• Amateur programs often lack the budgets for multi-camera rigs, operators, or analyst review, while commercial statistics products commonly retain a human role in closing the attribution gap.The paper treats manual review as a measurable method, not as ground truth assumed to be correct.Source: §1 · pp. 2, 4
• Across sports, adding candidate actors weakens identity attribution: court geometry can identify a side in tennis, but not a teammate within a volleyball team.The authors trace their own progression from two-actor boxing toward team sports, where they separate identity from action recognition in their model composition.Source: §1 · p. 3
• The evaluation uses 66 amateur matches filmed by a single fixed camera, with 46,648 hand-labelled contacts and no broadcast cues or second angle.The corpus is deliberately unlike broadcast footage and reflects the camera conditions common to school and club programs.Source: §1 · p. 3
• The authors favor testing a sport-specific approach in a data-rich, rule-constrained domain rather than assuming general intelligence will transfer uniformly across tasks.They frame this as a hypothesis to measure, not an established claim that domain-specific systems will outperform general ones.Source: §1 · pp. 3–4
• The paper asks which of four approaches earns its cost across match segmentation, contact spotting and classification, rally outcomes, team identification, player attribution, and totals.The approaches are prompted frontier vision-language models, small task-trained computer-vision models, self-supervised video world models, and manual annotation.Source: §1 · p. 4
• No single paradigm leads at every stage; frontier models segment rallies well, smaller trained models spot contacts more cheaply, rules recover rally outcomes, and automated identity attribution remains weakest.Action is a repeated motor pattern that temporal models can exploit, while a jersey number cannot be recovered if it is never legibly visible.Source: §1 · p. 4

2.1 From one body to a field of them

• Identity becomes harder as candidate actors increase: court geometry can distinguish opponents, but not similar-looking teammates, and crowded footage compounds the challenge.Published team-sport work reports limits in jersey readability [Koshkina and Elder, 2024], re-identification after actors leave the frame [Koshkina and Elder, 2025], and action attribution even with supplied tracks [Giancola et al., 2026].Source: §2.1 · p. 5

2.2 What the market actually ships

• Commercial systems that claim per-player statistics rely on human input, fixed capture setups, pre-registered rosters, or wearable sensors rather than an unprepared ordinary camera alone.Vendor guidance also emphasizes camera position and readable jersey numbers, while some products require customers to submit rosters or distinguishable numbers [Hudl, 2026b; Balltime, Hudl, 2026].Source: §2.2 · p. 6

2.3 Video understanding beyond sport

• The nine commercial readers score 0.46–0.59 on identity accuracy, and single-pass systems find only about a quarter of match events, consistent with the broader benchmark gaps.The same model families are evaluated on the authors’ footage and labels, rather than inferred from public leaderboard scores alone.Source: §2.3 · pp. 7–8
• General video benchmarks show a spread, not a ranking: coarse-grained tasks score highly, while fine-grained temporal and multi-turn tasks retain wide gaps.The table cautions that scores are not comparable across benchmarks because tasks and scoring rules differ.Source: §2.3 · p. 7, Table 1 · p. 8

3 Corpus and Setting

• 66 amateur matches contain 46,648 human-labelled contacts, including team, contact type, and usually the numbered player responsible.The held-out matches span multiple gymnasiums, camera positions, and years of footage.Source: §3 · p. 9
• Typical footage shows several players at once: median player height is under 200 pixels, with jersey digits typically only 15–30 pixels tall.Digits become difficult to distinguish from neighbouring numbers before reaching the lower end of their observed size range.Source: §3 · p. 9
• Camera angle can matter more than player size for identity: a 148-pixel sideline match scored 0.340, while a 93-pixel end-line match scored 0.418.The end-line view presents jersey numbers face-on, whereas the sideline view foreshortens them.Source: §3 · p. 9
• Contacts are labelled to one-second resolution, so evaluation matches predictions within a tolerance window; held-out matches were frozen and never used for development or tuning.The authors also report that most contacts have an acoustic onset within a fraction of a second of the labelled time.Source: §3 · p. 9

4 From Video to a Box Score: Seven Stages

• Segmentation divides continuous footage into rallies, and boundary errors can misattribute or lose an entire sequence of contacts.Event spotting then locates individual ball contacts within each segmented rally, typically to roughly one second.Source: §4 · p. 10
• Action classification labels each spotted contact using motion and context, while outcome determination identifies the point winner and any required terminal-event type.The action vocabulary is fixed by volleyball; a kill is an attack that wins the rally without being returned, while an assist is conventionally credited to the set preceding a kill.Source: §4 · p. 10
• Team assignment identifies which side made a contact, and player attribution identifies the specific player; mistakes at either stage undermine downstream per-team and per-player statistics.Player attribution is the hardest stage by a wide margin.Source: §4 · p. 10
• Aggregation sums attributed events into team and individual totals, but remains a measurable source of error even when its inputs are perfect.Earlier errors propagate to later stages, while success at one stage does not guarantee success at the next.Source: §4 · p. 11
• Sport rules can supply some answers independently of video, especially rally outcomes, while player identity has no comparable rule-based shortcut.The paper organizes stage-level reporting around how much each answer depends on domain rules versus visual perception.Source: §4 · p. 11

5 Four Approaches

• Prompting evaluates frontier vision-language models in single-call and agentic settings, where models may request context or reconsider answers before committing.Cost is reported as metered spend at providers’ published rates for an identical workload.Source: §5 · p. 11
• Trained specialists combine person detection and tracking with small task-specific models and symbolic sport rules; temporal models use appearance and motion over short frame windows.Components are on the order of a few million parameters, trained on labelled data and evaluated on a frozen split.Source: §5 · p. 11
• World-model probes test whether frozen self-supervised video or image features already contain fine-grained player-attribution information, without task-specific backbone training.Only a lightweight classifier is trained on top of the extracted features.Source: §5 · pp. 11–12
• Human annotation is treated as a method with measurable cost and latency, not as free, instant, or error-free ground truth.The authors do not independently measure manual annotation accuracy against their corpus.Source: §5 · p. 12
• Where feasible, the approaches receive the same footage, tracks, and candidate crops; comparisons with structurally different inputs are explicitly not treated as controlled.For example, agentic prompting uses raw video while trained specialists use cropped, tracked sequences.Source: §5 · p. 12

6.1 Segmentation and event spotting: why this has to be temporal

• Play boundaries require integrating motion and event density over several seconds, rather than classifying a single frame.Frame differencing, scoreboard flicker, and resetting players undermine simple deterministic boundary rules.Source: §6.1 · pp. 13–14
• Fixed motion thresholds and audio signals cannot reliably segment rallies because dead-time movement can resemble live play and audio cues vary by action.The best fixed threshold in one 40-second window still misclassified roughly half of live-versus-dead time; audio onset strength was weakest for common soft contacts such as digs.Source: §6.1 · p. 14, Figure 8 · p. 16, Figure 9 · p. 17
• Sensitivity analysis identifies costly error classes, but not which system component caused them or whether a fixed rule can solve them.The authors’ hand-engineered replacements did not improve on their existing segmenter, which was already near the task’s ceiling.Source: §6.1 · p. 15
• A prompted frontier model matches human-annotated play boundaries closely, while a small trained boundary model performs substantially worse.This is the study’s one stage where prompting a general model is unambiguously the right choice.Source: §6.1 · pp. 15–16
• A compact trained contact spotter outperforms frontier prompting at roughly one hundredth of a cent per match rather than several dollars.Contact spotting requires recognizing a brief kinematic signature, favoring dense short-window motion features over coarse video sampling.Source: §6.1 · pp. 16–17

6.2 Outcome, without seeing the ball

• Observing who serves next recovers the previous rally’s winner in essentially every observed transition, outperforming attempts to perceive the outcome from the ball.Every rally in the corpus began with exactly one serve.Source: §6.2 · p. 17

6.3 Team assignment

• Observing the server’s physical position raises team assignment from coin-flip performance to accuracy in the low-to-mid nineties across held-out matches.This avoids assuming which physical court side belongs to each team.Source: §6.3 · p. 17
• A side-assignment error can reverse every contact’s team label while leaving combined two-team totals plausible.The crowded-scene example shows thirteen candidate tracks in one frame, with several pairs overlapping by more than two thirds of their area.Source: §6.3 · p. 17, Figure 10 · p. 18

6.4 Identity: where every automated method converges

• Identity is the hardest stage; nine commercial readers span more than two orders of magnitude in list price but roughly ten percentage points in accuracy, and the most expensive is not most accurate.The needed digit shape occupies on the order of twenty pixels, which the passage says is not reliably present in the input.Source: §6.4 · p. 18
• Fine-tuning on labelled contacts does not close the identity gap, and the initial frozen-vision configuration scored 0.197 versus 0.298 for the zero-shot baseline while abstaining on nothing.The reported interpretation is that fine-tuning shifts guesses without meaningfully improving whether the digit is visible.Source: §6.4 · pp. 18–19
• A frozen video-world-model representation achieves identity accuracy only narrowly above a baseline using candidate bounding-box coordinates alone.The passage relates this result to earlier comparisons and to literature describing fragmented local structure rather than the spatial detail needed for fine-grained identity [Kodathala and Vunnam, 2025] [Mur-Labadia et al., 2026].Source: §6.4 · p. 19
• Composed classical detection and tracking with task-specific specialists performs best for identity, but still leaves a substantial share of contacts unresolved.Doubling working resolution produced a larger accuracy gain than the architectural changes tested, including deeper temporal models and ensembles.Source: §6.4 · p. 19

6.5 Whole-match reasoning: where a frontier model actually helps

• Holistic, full-resolution agentic reasoning raised team-level event detection from roughly a quarter of events found to a little under half, compared with short, low-resolution single-pass clips.The evaluation allowed re-examination of ambiguous moments and explicit tracking of match state.Source: §6.5 · p. 20
• Reasoning-enabled jersey-reading configurations reached the high fifties, but no tested configuration made per-player reads reliable, and the evaluated model releases were specific and dated.The passage bounds the identity gap by information optically present in the source footage, not reasoning capacity.Source: §6.5 · pp. 21–22

6.6 Why context helps action, and does not help identity

• A temporal window helps action recognition because sporting actions are repeated motor sequences whose identifying signal is distributed across time, unlike identity, which remains tied to the face or jersey.The passage connects this distinction to temporal models classifying actions more reliably than single-instant reasoning.Source: §6.6 · pp. 22–23
• End-line views keep jersey numbers close to fronto-parallel, while sideline views make far-side numbers increasingly edge-on, an effect resolution cannot correct.The passage identifies viewing angle together with actor height as a better predictor of identity accuracy than height alone.Source: Figure 16 · p. 23
• A jersey number is static, so viewing a whole play helps only by searching for the rare frame where the player faces the camera and the number is unobscured.Across a play, legibility rose roughly fourfold for attacks and close to certainty for sets.Source: §6.6 · p. 23
• Adding parameters, candidate frames, or independently trained heads moved identity accuracy less than changing what the model saw, so scale alone was not the useful lever.The passage contrasts those additions with the improvements associated with more informative input.Source: §6.6 · p. 24, Figure 18 · p. 24

6.7 Why accuracy differs so much across statistics

• Serve attribution reached a player-level rank correlation of 0.81 across 303 true serves and correctly named the leading server in four of five matches, while kill attribution reached 0.02 across 98 true kills.The passage attributes the contrast to rotation and service-zone rules narrowing serve candidates, unlike contested kills with no comparable rule-based narrowing.Source: §6.7 · pp. 24–25
• Predicted team kill totals track true totals closely even as player-level kill-count rank correlation is near zero, so credible team sums can conceal noisy attribution.The section reports statistics at team, player-rank, and leading-player levels to expose this discrepancy.Source: §6.7 · pp. 24–25
• A rule inserting a plausible terminal attack recovers many missed events but does not meaningfully improve attribution of the acting player.The passage says the recovered events are disproportionately ambiguous both as events and as to who acted.Source: §6.7 · p. 25, Figure 20 · p. 26

6.8 Abstention: agreement, not confidence

• The paper separately decomposes attribution failures by selection errors, passed-over candidates, illegible actors, and the absence of any viable candidate..Source: Figure 21 · p. 26
• Self-reported confidence was statistically indistinguishable from overall accuracy, whereas agreement among independent readers identified contacts accurate enough to present and disagreements suited for human review.The passage recommends basing abstention on disagreement rather than a model’s stated confidence.Source: §6.8 · pp. 25–26, Figure 19 · p. 25

6.9 Temporal resolution is set by the task, not the model

• Fine-grained player attribution requires higher temporal sampling than coarse segmentation, but commercial APIs use a fixed low rate regardless of the requested task.This sampling constraint is independent of source-video resolution and duration.Source: §6.9 · p. 27

6.10 A worked example: the same rally, three real requests

• The demonstration used one held-out 13.33-second rally with seven human-labelled contacts and changed frame rate or prompt across requests.The first two requests used the same event-list prompt; the third changed only to a generic description request.Source: §6.10 · pp. 27–28
• At one frame per second, the model found zero contacts in a rally containing seven; at three frames per second, it found ten, including three extra events.At the higher rate, timestamps drifted by roughly a second by the end of the clip.Source: §6.10 · pp. 27–28
• With the same 40 frames, a generic description prompt produced fluent prose but no timestamps or countable event structure, and invented a blocked point and celebration.The event-list prompt instead returned a recognizable, though imperfect, contact sequence.Source: §6.10 · p. 29

• A single match-level accuracy can conceal poor performance on rare statistics because frequent serves dominate pooled results, so each statistic is reported separately at multiple aggregation levels.The reported spread includes both team-level and player-level results.Source: §6.11 · p. 29
• Each event is scored against its own matched ground-truth event, so a correct contact cannot offset an incorrect one elsewhere.Aggregation happens only after event-level scoring.Source: §6.11 · p. 29
• Coverage and precision are reported separately because systems may abstain, and each abstention tier is kept distinct rather than averaged.Coverage is the share of contacts answered; precision is correctness among answered contacts.Source: §6.11 · p. 30
• Stages are scored conditional on preceding success; identity is scored only for action-matched contacts whose human labels include a jersey number.The protocol also reports per-player rank correlation, resamples whole matches, and presents macro alongside volume-weighted statistics.Source: §6.11 · p. 30

7.1 Where identity is lost

• The largest identity-loss category occurs when the correct player was legibly identified and offered as a candidate, but another candidate was selected.Other losses arise when no viable track exists or the acting player’s number is never legible during the play; higher working resolution measurably improved the selection-failure category.Source: §7.1 · p. 31
• Identity accuracy varies across matches without a monotonic relationship to camera axis or median actor height alone; their combination is a better predictor.The table covers five held-out matches with per-player jersey ground truth; a sixth match without that ground truth is excluded.Source: Table 6 · p. 32

7.2 Why a top-down architecture outperforms end-to-end reasoning here

• Detecting people, tracking them, then selecting which track acted bounds the choice to visible candidates and lets the system read identity at the player’s most legible moment.This decomposed design also makes errors attributable to a pipeline stage; the paper relates its structure to specialized detection and track-then-read approaches [Al-Hamadani, 2025] [Koshkina and Elder, 2024].Source: §7.2 · pp. 32–33
• Iterative zooming or visual self-correction does not reliably recover missing visual information, and reported gains from those strategies can largely reflect existing model capability [Ma et al., 2026] [Tripathy and Krishnan, 2026].The paper’s prompting measurements likewise do not recover information absent from the footage.Source: §7.2 · p. 33

7.3 Why global rules help and local chaining hurts

• Global rules such as one serve per rally help, whereas local chaining constraints reduce attribution accuracy because an upstream error can remove the correct next candidate.Global constraints do not depend on any one earlier decision being correct.Source: §7.3 · p. 33

7.4 Why a fixed threshold does not travel across cameras

• A fixed pixel-height threshold does not transfer cleanly across cameras because sensor, lens, distance, and viewing angle change how legible a jersey number appears.The paper notes related camera-dependent variation in pose estimation and motion reconstruction.Source: §7.4 · pp. 33–34
• The deployment boundary is camera-specific: the paper reports accuracy against its described camera conditions rather than as an unconditional number.The authors connect this caution to commercial single-camera motion-measurement products outside sport [Insolvency Insider, 2025].Source: §7.4 · p. 34

• Concurrent processing and reuse of completed match results support throughput above serial processing, with a new match completing well within match length.Prompting-based costs summed across seven stages are several times higher than the composed system’s full-match cost, with a spotting-stage difference of several orders of magnitude and no corresponding accuracy advantage at most stages.Source: §8 · p. 34
• The interface presents agreed attributions as confirmed, disputed ones as candidates for human confirmation, and unresolved contacts for review.Its abstention tiers use inter-reader agreement rather than a model’s stated confidence; a correction propagates across the player’s tracked contacts.Source: §8 · p. 34
• Coaches review a minority of contacts, concentrated in the system’s low-confidence tier, rather than annotating the whole match.The authors report that this review fraction shrinks as footage quality improves, while explicitly not claiming that the system eliminates human review.Source: §8 · p. 34

9.1 The same fix, four times

• Rule-based facts bring outcome determination, team assignment, and segmentation close to ceiling, while identity remains the exception because volleyball’s rules do not identify which player touched the ball.The discussion attributes identity’s lower accuracy to the absence of a rule-based structural fact that can substitute for observing the player.Source: §9.1 · pp. 34–35

9.2 What transfers across sports, and what does not

• Rule density predicts transfer: discrete, rule-governed sports offer more structure than continuous-play sports, so the volleyball results do not transfer uniformly.The discussion presents the stage-by-stage framework as generalisable, while noting that sport-specific numbers do not necessarily transfer.Source: §9.2 · p. 35

9.3 The single-actor-to-field-sport axis, revisited

• Viewing angle compounds resolution and the lack of a second vantage point, making amateur identity attribution harder than broadcast validation alone may suggest.A sideline view foreshortens jerseys; an end-line view presents their faces more legibly.Source: §9.3 · p. 35

9.4 What this is not: a review or judging system

• The system is retrospective and advisory, not a real-time or binding referee-review system requiring certainty appropriate to contested decisions.Its outputs may be corrected by a coach; they do not adjudicate a match.Source: §9.4 · p. 36

9.5 Why amateur sport needs this to be real work, not a demonstration

• For amateur athletes without paid analysts, an uncertain system that flags unresolved statistics can be more useful than having no statistics or inconsistent handwritten tallies.The discussion treats reporting negative results as useful to coaches deciding which numbers to trust.Source: §9.5 · p. 36

9.6 What a commercially useful account of this problem can and cannot disclose

• The paper reports accuracy, costs, negative results, and architecture, but withholds commercial prompts, thresholds, configurations, and calibration choices.The authors frame measured comparisons and the explanation of identity difficulty—not reproduction of calibrated commercial internals—as the contribution.Source: §9.6 · p. 36

Corpus scope

• The corpus covers one sport, predominantly youth and school teams in one region, and a limited set of camera setups; its Section 6 numbers are volleyball-specific, not sport-agnostic constants.The stage-by-stage framework and discussion of transfer are presented as generalisable, but the measured numbers are not.Source: §10 · p. 37

Attribution limits

• Individual-player kill attribution remains unresolved and coach-confirmed, while set attribution is also weak and not shipped.Team-level totals should not be taken as evidence that per-player kill statistics are production-ready.Source: §10 · p. 37

Evaluation sample

• Some per-match results use few held-out matches because independently verified per-touch ground truth is expensive and scarce.The authors provide per-match breakdowns so readers can assess the statistical weight rather than rely on a pooled figure.Source: §10 · p. 37

Method comparison

• Neither prompting nor trained-specialist results are hard ceilings: API sampling and context limits constrain prompting, while labelled-data availability constrains specialists.The authors say differently engineered prompting or a larger specialist trained on more data could plausibly do better.Source: §10 · p. 37

Vendor comparison

• The study evaluates methods, not named vendors, because vendors do not publish the per-touch, per-player ground truth required for this comparison.Vendor statements about required conditions do not establish vendor accuracy.Source: §10 · p. 37

Ethics

• The study uses pre-existing licensed footage of identifiable minors and young adults, publishes no names, and obscures faces and venue-identifying details in figures.The authors did not seek a formal institutional review determination, describing the work as secondary analysis of licensed recordings.Source: Statements · p. 37

Data availability

• The commercially licensed corpus of 66 matches and per-touch annotations is not released, but its composition, evaluation protocol, and quantitative results are reported.The paper says these materials are provided to support assessment and further work without the underlying video.Source: Statements · p. 37

Code availability

• The production system’s specific configuration is withheld as commercial operating configuration, while component roles, training-data sources, and evaluation protocol are described.The authors state that comparative findings can be independently replicated using non-proprietary tooling.Source: Statements · p. 38

Competing interests

• All authors work in R&D for the company whose system and data licence are involved, and the paper identifies this as a competing interest.The authors say they mitigate it by reporting negative and null results, avoiding named-competitor accuracy comparisons, and applying the same protocol across paradigms.Source: Statements · p. 38

Use of AI tools

• Language models were evaluated as a paradigm and separately used for manuscript editing and literature checks, with citations checked by the authors.The authors take responsibility for the paper’s content and claims.Source: Statements · p. 38

Funding

• The work was internally funded by Sports Vision, Inc., with no external funding.Source: Statements · p. 38