A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions

1 Introduction:

• Player and ball detection and tracking face appearance similarity, occlusion, camera motion, field calibration, broadcast editing, low resolution, blur, and unpredictable movement; the survey covers classification, trajectory prediction, and team strategies.Simultaneously tracking players and the ball is especially challenging when they move unpredictably, the ball changes players, or severe occlusion occurs.Source: §1 · p. 2
• The review examines prior computer-vision approaches, their key points and weaknesses, and whether current implementations can support real-time sports video analysis.It also frames higher-level semantic interpretation as useful when analysis requires real-time operation and minimal human intervention.Source: §1 · pp. 2–3

1.1 Features of the Proposed Review

• The proposed review surveys sports computer-vision applications and establishes a pathway for future research, including statistics across sports and AI algorithms used in sports analysis.Its application scope includes player detection and classification, player or ball tracking and trajectory prediction, team strategies, and event classification.Source: §1.1 · pp. 6–7
• The review provides a roadmap for selecting and evaluating AI algorithms and includes publicly available datasets for different sports..Source: §1.1 · p. 7
• It discusses GPU-based embedded platforms for real-time object detection and tracking, sports-vision applications, and possible research directions.The stated aim for edge-device platforms is to improve performance and accuracy.Source: §1.1 · p. 7
• Sections 8 and 9 cover potential research directions and challenges in sports studies, followed by the review’s concluding considerations.Source: §1.1 · p. 7

• Detecting player positions at a given time is a basic step in tracking players.Source: §2 · p. 7
• Analyzing individual player movements and real-time team formations can give coaches real-time insight to improve team performance.Source: §2 · p. 7
• The surveyed articles were selected by peer review from high-impact online sources and span detection and tracking, classification, performance analysis, gesture recognition, highlights, and score updates.Source: §2 · p. 7
• Figure 3 summarizes sports research publications from the past five years considered in the review.Source: §2 · p. 8, Figure 3 · p. 8
• Figure 4 presents studies across sports and applications including player and ball detection or tracking, trajectory prediction, classification, and video summarization.Source: §2 · p. 8, Figure 4 · p. 8

• Play-field extraction aims to separate field from non-field areas and filter redundant pixels such as grass and court lines, simplifying player/ball tracking, event extraction, and pose detection.Source: §3 · p. 9
• Field color can be confused by stadium surroundings, lighting, weather, viewing angles, shadows, or matching player clothing, so color statistics may need updating during play.Source: §3 · p. 9
• Morphological operations and background subtraction can separate regions or generate foreground masks, but neither detects playfield lines.Source: §3 · pp. 9, 11
• Color-based extraction uses a dominant field color and illumination-resistant image features transformed to HIS [13, 14, and 15], YCbCr, or normalized RGB.Source: §3 · p. 10
• For field/court tracking, camera calibration and an appropriate number of cameras are necessary; soccer systems track players and the ball using multiple cameras, while bird’s-eye views support trajectory estimation and team classification.Source: §3 · p. 10
• Training on labeled playfield lines, advertisements, and non-playfield regions enables their detection and classification, reducing false positives and false negatives.Source: §3 · p. 11

4.1 Basketball

• Basketball methods estimate ball trajectories from player pose [37], assess behavior from referee signals, and extract game highlights.Source: §4.1 · p. 12
• Team behavior is characterized through space-creation dynamics and defensive play that counteracts them.Source: §4.1 · p. 12
• Court locations are used to predict player movement and generate three-dimensional ball trajectories, while reconstructing and tracking shooting motion remains a research need.The review describes three-dimensional ball-trajectory generation as complicated.Source: §4.1 · p. 12
• Rapidly changing play and complex data structures hinder analysis, so the review identifies real-time gameplay analysis as necessary.Source: §4.1 · p. 12
• One basketball result entry reports an accuracy of 76.5%.Source: §4.1 · p. 14
• A basketball classification approach achieved 69% classification.Source: §4.1 · p. 14
• Future basketball research may consider defensive strategies such as pick-and-roll and pick-and-pop.Source: §4.1 · p. 14
• A basketball method performed well in convergence rate and achieved a final AUC of 91%.Source: §4.1 · p. 14
• Deep learning models outperformed conventional models for predicting NBA player positions, but a proposed model failed at trajectory prediction.Source: §4.1 · p. 14

4.2 Soccer

• Soccer tracking is challenged by near-identical jersey colors, occlusions, and player movement; systems track players and the ball to derive positions and trajectories.Source: §4.2 · p. 15
• Soccer vision systems support player and team performance analysis, including activity measurement, shot detection, pass classification, key-event extraction, tactical analysis, and highlight generation.Source: §4.2 · pp. 15–16
• Soccer methods have reported limits including failure under severe or same-color-player occlusion, missed ball tracking at critical passes, and difficulty identifying shot-involved players or estimating formations early.Source: §4.2 · p. 16
• Soccer reports include 97.56% real-time accuracy, ANN win/loss predictions of 83.3%/72.7%, action-recognition F1 of 52.8% for 6 classes, and VGG-M CNN accuracy of 87.45%.Source: §4.2 · p. 17
• The LSTM achieved 53% f1-score, but failed to predict players amid unexpected speed or direction changes and was limited in scalability.Source: §4.2 · p. 17
• Formation detection reached a maximum accuracy of 96.8%, but evaluation was limited to one match and could not assess tactics across matches.Source: §4.2 · pp. 17–18
• Real-time formation visualization limits depiction of non-trivial schemes, while state-of-the-art tracking algorithms could improve tactical performance.Source: §4.2 · p. 18
• Combining Spatial Constellation features achieved 82% accuracy, but the model failed on invisible players, whose position prediction could improve the features.Source: §4.2 · p. 18
• A pass-evaluation model achieved 90.2% accuracy; quality factors include execution difficulty, strategic value, and risk, while sequence ratings require pass order.Source: §4.2 · p. 18
• Static RNN achieved 85.5% test accuracy, but the model was not robust and did not detect the small ball in far-view frames.Source: §4.2 · p. 18
• The audiovisual methodology achieved 98% precision, 97% recall, and 98% f1-score, but audiovisual features failed to recognize quick changes in team tactics.Source: §4.2 · p. 18
• The cited passages indicate that location information was represented with a heat map, and the classification task achieved 87% accuracy; they do not support the stated k-NN result or class count.Source: §4.2 · p. 18

4.3 Cricket

• Reviewed applications include batting-shot and bowling analysis, ball-trajectory estimation, foot-placement decisions, commentary and umpire-decision classification, performance and match prediction, and video summarization.The section also describes summarization as selecting exciting events to create a compact version of the original video.Source: §4.3 · p. 19
• For four-class classification, the decision-tree methodology reports 96.82% precision, 94.83% f1-score, and 96.32% accuracy.The review notes low performance associated with the large number of trees and sensitivity to small changes in the decision tree.Source: §4.3 · p. 20
• Ball-by-ball outcome classification reaches a maximum 85% training accuracy and 74% validation accuracy, while lacking a standard cricket dataset.The review says the resulting accuracy is not considered satisfactory and calls for better accuracy.Source: §4.3 · p. 20
• Automated umpire-signal classification reports 94% accuracy with DCNN and 100% with Inception V3.The work proposes using artificial intelligence to automate scoring signals.Source: §4.3 · p. 20
• A bowling-classification model has 93.3% test accuracy, but its data cover left-arm bowlers and it misclassifies right-arm bowlers.The review also notes a lack of data for detecting spin bowlers.Source: §4.3 · p. 20

4.4 Tennis

• A batting-shot classifier identifies 20 classes and reports an average F1-score of 88%.The subsection also reports a 71% accuracy result based on statistics from 366 matches.Source: §4.4 · p. 21
• Ball-speed variation had feeble significance in influencing bowling performance, with a reported value of 3.394 and corresponding p-value 0.015.Source: §4.4 · p. 21
• Tennis ball- and player-tracking systems, including HawkEye[139, 140] and TennisSense, detect and track the ball/player, visualize the overall tennis match, predict trajectories of ball landing positions, and recognize player activity.The review frames automated tracking as a way to reduce human errors and extract statistics from the visual feed of the game.Source: §4.4 · pp. 21–22
• Tennis swing models can fail when movements are similar or players share movement patterns despite different styles, and one model lacks real-time learning and requires substantial computing time.The reported limitations also include difficulty classifying swing direction and non-linear cases.Source: §4.4 · p. 22

4.5 Volleyball

• Volleyball activity analysis uses 1-D space charts for patterns and point outcomes, but reported performance deviates across matches because training used those patterns and outcomes.The passages also note that the proposed method cannot handle multi-object tracking.Source: §4.5 · p. 23
• Volleyball studies report that continuous positional data can support tactic prediction and that temporal action localization may improve player tracking.The cited passages also note tracking failures, but do not preserve the condition under which they occur.Source: §4.5 · p. 24
• A volleyball method predicts ball trajectory 0.3 s in advance from body-position motion data, but records a large error.Source: §4.5 · p. 24

4.6 Hockey/Ice-hockey

• HD+SVM achieved the best reported accuracy, recall, and F1-score results in the hockey studies.The supplied text gives 77.24% and 69.23% before truncating the third value, so the F1-score value cannot be reported.Source: Table 7 · p. 25, §4.6 · p. 26
• Player identification achieved over 87% accuracy, although jersey-number classes 1–4 and split-diagonal numbers were misclassified.Source: §4.6 · p. 26
• A hockey model recognizes free hit, goal, penalty corner, and long corner, but similar activity patterns remain difficult to distinguish.Source: §4.6 · p. 26
• The hockey passages report 85% accuracy for a novel approach and a 67% F1-score for multi-labeled action recognition.Source: §4.6 · p. 26

4.7 Badminton

• Badminton vision supports training and match movement analysis, including action recognition, player-performance analysis, and shuttlecock detection and tracking.The review frames these technologies as supporting player training and performance improvement.Source: §4.7 · p. 27

4.8 Miscellaneous

• Across other sports, surveyed tasks include athletic player detection and tracking [194, 195], ball detection and tracking, pose detection, action and event analysis [224], performance analysis, referee gestures, and video highlights.Source: §4.8 · p. 28
• A player-tracking approach uses team-class and pose-guided partial features with temporal ID correlation in a K-shortest-path method to address identity switches.Source: §4.8 · p. 29
• 92% accuracy was reported for classifying ten types of environmental conditions.Source: §4.8 · p. 29
• The reported 95.8% method performs worse without gradual transitions.Source: §4.8 · p. 30
• 95.8% average accuracy was reported on a dataset of 20 videos spanning four sports categories.The method’s performance drops when gradual transitions are absent.Source: §4.8 · p. 30
• An unsupervised player-extraction method supports precise temporal segmentation of rally scenes and is robust to video noise, while camera shaking and occlusions remain challenges.Source: §4.8 · p. 30
• A template-matching algorithm is used for overlapping-player tracking, and the review suggests replacing it with state-of-the-art deep-learning methods for better results.Source: §4.8 · p. 30
• 94% accuracy is reported for overlapping two-player cases, although player overlap and similar features can reduce tracking performance.Source: §4.8 · p. 30
• The method achieves better tracking accuracy than TMS and CMS algorithms, but tracking can fail when target scale changes or remains unaccounted for.Source: §4.8 · p. 30

4.9 Overview of Machine Learning/Deep Learning Techniques

• Figure 10 presents deep-learning techniques for classification -[245], detection [246], unsupervised learning [247], tracking [249]-[258], and trajectory prediction -[266].Source: Figure 10 · p. 33, §4.9 · p. 33
• Figures 8 and 9 provide roadmaps for machine-learning and deep-learning algorithm selection, training, evaluation criteria, and required tools.Source: Figure 8 · p. 32, Figure 9 · p. 32, §4.9 · p. 33

• Shared datasets support transparent algorithm comparisons on common data and reduce the effort of capturing and annotating large video collections.Source: §5 · p. 33
• Datasets include moving-camera videos or still images for player-action recognition and event detection, as well as static-camera setups for tracking, pose estimation, and event summarization.The section also notes a dataset focused on spectator actions rather than players.Source: §5 · p. 34
• ISSIA annotates ball, player, and referee positions in each video from each camera [274].Source: §5 · p. 34, Figure 11 · p. 34
• TTNet annotations mark ball bounces, ball hits on the net, and empty events [268].Source: §5 · p. 35, Figure 12 · p. 35
• APIDIS videos use seven cameras positioned above and around the court, with annotations for player positions, referee movements, baskets, and ball position [275].Source: §5 · p. 35, Figure 13 · p. 35

• NVIDIA Jetson modules provide low-power, high-performance support for AI visual recognition and are configured with OpenCV, cuDNN, CUDA Toolkit, L4T, and TensorRT.Table 11 compares Jetson modules with GPU-constrained devices and possible deep-learning algorithms, but the supplied passages do not provide its cell values.Source: §6 · p. 38, Table 11 · p. 38
• A stereo vision system tracked a golf ball traveling at 360 km/h under indoor lighting using an FPGA board with an ARM CPU.Source: §6 · p. 38
• P R Kamble et al. tracked a soccer ball with a deep-learning approach on an NVIDIA GTX1050Ti, while L Chen et al. used a GTX 1080 Ti with CUDA 9.0 and Caffe to analyze basketball technical features.Source: §6 · p. 38
• FPGA applications in sports include 3D motion capture, object movement analysis, and image recognition, with Table 12 listing study results by performance measures.The supplied passages describe the table's scope but do not provide its performance values.Source: §6 · p. 39, Table 12 · p. 39

7 Applications in Sports Vision

• Basketball movement analysis reports 92.30% accuracy, while personalized apps, offers, and live game updates can engage digitally connected fans and create revenue opportunities.Source: §7 · p. 40

7.1 Chabot’s& Smart Assistants

• NHL and NBA teams use virtual assistants to answer fan questions about ticketing, arena logistics, parking, and game information.Source: §7.1 · p. 40

7.2 Video highlights

• AI can address demands for analysis, commentary, action-packed highlights, and behind-the-scenes content across devices, creating an engaging viewing experience that attracts viewers.Source: §7.2 · p. 41

7.3 Training & Coaching

• AI training applications combine game and training data with coach and sports-scientist knowledge, updating that base to educate coaches, scientists, and athletes and improve performance.Source: §7.3 · p. 41

7.4 Virtual Umpires

• Player-requested reviews involving other parties can consume time and interrupt game momentum, motivating faster AI-assisted officiating.Source: §7.4 · p. 41
• AI-supported cameras could leave umpires focused on player behavior; computer vision can detect tennis-ball placement and speed, eliminating line umpires, while earpieces and glasses could deliver instant decisions without reviews.Source: §7.4 · p. 41

7.5 AI Assistant Coaches

• An AI assistant coach could run alongside gameplay to dynamically predict and create strategies, potentially gaining teams an edge over coaches relying on prior data and experience.Source: §7.5 · p. 42

7.6 Available Commercial Systems for Player and Ball Tracking

• Hawkeye provides ball tracking in cricket, tennis, and soccer, while STATS SportVU and ChyronHego TRACAB provide player tracking for performance analysis and coach training.Source: §7.6 · p. 43

8 Research Directions in Sports Vision

• The review groups future directions by task-specific needs, including player, ball, and referee detection and tracking, pose estimation, and trajectory prediction.These directions are presented as potential research areas for computer vision and video processing across sports.Source: §8 · p. 44, Figure 18 · p. 44
• Soccer strategy analysis can improve by labeling large spatio-temporal datasets and classifying actions into subtypes; trajectory analysis and temporal information can sharpen team-tactics insights and reveal pass events.Accurate pose detection remains challenging for distinguishing running, jumping, and walking and for handling occlusion or identity switches.Source: §8 · p. 45
• Cricket analysis needs to model batting stance and shot choice by bowling type, improve umpire-pose classification, and precisely detect bat contact and run-outs.Spin-ball trajectory prediction is another open direction, for which the review proposes larger labeled datasets and state-of-the-art algorithms.Source: §8 · p. 45
• Badminton research should improve action recognition with larger datasets and end-to-end fine-tuning, while linesman systems need to track shuttle paths despite distant views, illumination changes, and shuttle speed.Path tracking can help referees decide whether a shuttle landed in or out.Source: §8 · p. 46, Figure 23 · p. 46

8.1 Open Issues and Future Research Areas

• Camera calibration and viewpoints remain unresolved, and ball detection and tracking are open problems because balls are small, fast, and move unpredictably relative to players and the field.Ball detection supports recognition of events such as goals and possession.Source: §8.1 · p. 46
• Player and ball tracking remains an open research area: fast, frequent movement and similar jersey colors pose issues; linear-motion algorithms perform poorly, while appearance models address association but fail under severe occlusion or appearance ambiguity.Source: §8.1 · p. 46

8.2 Future Research Trends according to Methodologies in Sports Vision

• Sports algorithms depend on dataset annotations, losses, and evaluation metrics, while real-time deployment must balance accuracy, computation speed, and model size.Identity-aware deep representations are proposed for tracking players, referees, and goalkeepers; ball-state recognition remains fragile across ball sizes, shapes, velocities, and environments.Source: §8.2 · p. 47

8.3 Different Challenges to Overcome in Sports Studies

• Jersey-number recognition is harder for hockey and American-football jerseys, while action recognition is nonlinear and requires representations that capture video’s temporal structure.The review points to bounding-box and digit-recognition methods for jerseys and feature alignment and discriminative video representations for actions.Source: §8.3 · p. 47

9 Conclusion

• The review covers player and ball tracking, trajectory prediction, skill and strategy analysis, and object detection and classification across sports.It also provides publicly available sport-specific datasets.Source: §9 · p. 48
• It discusses classical and AI techniques, their performance and suitability, GPU workstations, embedded platforms, sports AI applications, research directions, challenges, and trends.Source: §9 · p. 48
• Player tracking is challenged by similar appearances, blurry footage, occlusion, and unreadable jersey numbers, although computer vision is presented as a possible solution.These difficulties can impede reliable identification in sports footage.Source: §9 · pp. 48–49
• Jersey recognition is harder for sports with massive, sharply contoured jerseys, while sport-specific player appearances limit cross-sport model transfer.The review suggests bounding-box and digit-recognition methods, and fine-tuning with samples from multiple sports.Source: §9 · pp. 48–49
• Optical tracking can help with occlusion and similar appearances, but fans in team uniforms can cause false positives; playground or camera estimates with player positions may help.Source: §9 · p. 49
• Action recognition is treated as a nonlinear problem requiring aligned discriminative features, temporal structure, and salient frame-region analysis.Source: §9 · p. 49
• Kalman and particle filters handle ball size, shape, and velocity but can lose tracks after occlusion; trajectory methods handle occlusion and missing or merged balls but fail on size and shape variation.Data association suits small balls in small courts such as tennis, not sports such as basketball, soccer, or volleyball.Source: §9 · p. 49
• Ball tracking and semantic scene understanding remain open problems amid rapid movement, similar appearances, extreme aspect ratios, and frequent occlusion.The review calls for better AI algorithms, ball databases spanning sizes and shapes, and advance analysis of formations, passes, shots, goals, and game strategies; cross-sport benchmarks are difficult because datasets, ground truths, capture devices, and conditions vary.Source: §9 · pp. 49–50

Abbreviations:

• BEI-CNN stands for Basketball Energy Image - Convolutional Neural Network.Source: Abbreviations · p. 67
• Deep-SORT stands for Simple Online Real Time Tracking with Deep Association.Source: Abbreviations · p. 67
• Faster-RCNN stands for Faster-Regional with Convolutional Neural Network.Source: Abbreviations · p. 67
• GRU-CNN stands for Gated Recurrent Unit - Convolutional Neural Network.Source: Abbreviations · p. 67
• Mask R-CNN stands for Mask Region-based Convolutional Neural Network.Source: Abbreviations · p. 68
• R-CNN stands for Region-based Convolutional Neural Network.Source: Abbreviations · p. 68