PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Large Models in Sports

Yichen Xu, Jianzhe Ma, Chuhan Wang, Zhonghao Cao, Liangyu Chen, Wenxuan Wang, Qin Jin

arXiv:2608.14377v1cs.CLcs.CV

TL;DR

Large models are rapidly entering sports, but the field lacks a consolidated view of their applications, datasets, benchmarks, and challenges. This survey organizes the literature across six stakeholder groups and 19 tasks, analyzes datasets and benchmarks, and identifies challenges and future directions for sports intelligence.

  • Problem

    Rapidly expanding sports applications lack a consolidated analysis of large-model tasks, datasets, benchmarks, and challenges.

  • Method

    The survey synthesizes 241 papers, organizing applications across six stakeholder groups and 19 tasks while analyzing datasets, benchmarks, challenges, and future directions.

  • Results

    The survey establishes a structured taxonomy of six stakeholder groups, analyzes relevant datasets and benchmarks, and identifies fundamental challenges in large models for sports.

  • Takeaways & Limitations

    The survey provides a framework and practical resource for future research and development of large-model-driven sports intelligence.

  • Takeaways & Limitations

    The survey may miss recent progress, non-English research, and in-depth coverage of underrepresented or niche sports because of its scope and literature-selection protocols.

Abstract

from arXiv · show

Sports have witnessed growing global enthusiasm in recent years, serving as a vital force for physical health, cultural exchange, social connection, and economic growth. The rapid advancement of large models, particularly (multimodal) large language models (M)LLMs, has demonstrated transformative potential to reshape sports understanding, analysis, and interaction across diverse domains. This paper presents a comprehensive survey of large models in sports, including (i) an overview of tasks and applications across different participant groups; (ii) a detailed analysis of sports-related datasets and benchmarks; and (iii) a critical discussion of current challenges and future directions. Our goal is to establish a foundation for advancing research and practical development of large-model-driven sports intelligence. An open-source GitHub repository is maintained at: https://github.com/Road2Redemption/Awesome_Large_Models_In_Sports1.

1 Introduction

The survey examines how large language and multimodal models are creating new opportunities and challenges for sports through language and multimodal intelligence. It systematically reviews applications across six stakeholder groups, sports datasets and benchmarks, current challenges, and future directions, based on 241 core papers.

  • Motivation: Sports participation and industry growth increasingly depend on massive data, while AI has accelerated this data-driven development.The introduction identifies processing and generating sports-related language as a pivotal capability in this transformation.
  • Research gap: Existing reviews cover exercise recommendations, sports science and medicine, the sports industry, and language or multimodal datasets, but remain incomplete for large models in sports.Earlier sports-AI research also addressed sports data processing and video analysis.
  • Large-model transition: Large language and multimodal models such as GPT-4 and Gemini extend sports intelligence by generating language and processing multiple data modalities.Their emergence brings both new opportunities and challenges to the sports domain.
  • Methodology: The authors collected 241 core academic papers through systematic snowballing and PRISMA-aligned searches covering January 2020–July 2025.The search proceeded through iterative forward and backward citation searches from prior review papers.
  • Survey scope: The survey organizes applications across 6 sports stakeholder groups, then analyzes datasets and benchmarks before discussing challenges and future directions.The six groups are athletes and trainers, coaches and educators, referees, researchers, fans and social media, and the sports industry.

2 Large Model Applications in Sports

Large models in sports are organized across 6 stakeholder groups and 19 tasks, with the survey analyzing task definitions, technical paradigms, and evaluation metrics. Applications span training, action understanding, officiating, commentary, education, management, and other sports-related activities, while unified evaluation and domain-specific reliability remain recurring challenges.

  • Overview: The survey categorizes sports applications into 6 stakeholder groups and 19 specific tasks, analyzing task definitions, technical paradigms, and common evaluation metrics.The taxonomy is presented as the organizing framework for the section.
  • Action Spotting and Recognition: On SoccerNet-v2, fine-tuned Soccer-CLIP achieves 75.7% t-AmAP, surpassing specialized Transformers at 73.1% and highlighting pretraining’s value for action spotting.Recent MLLMs use pre-training alignment for enhanced semantic understanding of fine-grained player movements and events.
  • Sports Action Quality Assessment: Specialized small-scale Transformers outperform general MLLMs on FineFS, scoring 0.88 versus 0.86 Spearman’s ρ through explicit audio-visual feature alignment.The result indicates that high-precision sports action scoring still benefits from domain-specific structural designs.
  • Sports Refereeing: Large models assist refereeing with question answering, captioning, and action recognition, while X-VARS applies QLoRA fine-tuning for rule-aware soccer video understanding.The approach represents an initial step toward explainable LLMs for sports refereeing.
  • Sports Commentary Generation: Commentary systems combine event descriptions, tactical analysis, and emotional insight using fine-tuned MLLMs or agentic prompting, but domain inaccuracies and absent unified metrics persist.Prompt engineering is used to improve factual accuracy, personalization, and engagement in generated narratives.

3 Datasets for Large Models in Sports

The section categorizes sports datasets for large models by design objectives and analyzes their distributions across sports, applications, modalities, annotation sources, and modeling paradigms. It identifies substantial coverage imbalances, scarce expert labels, and a predominance of generative tasks.

  • Dataset Taxonomy: Sports datasets are divided into task-specific datasets and sports understanding datasets, with analyses examining distributions across multiple facets.Task-specific datasets support six stakeholder groups and 19 tasks with annotations and evaluation metrics for training, fine-tuning, and assessment.
  • Dataset Distribution by Sport Type: 74 soccer datasets contrast with only one dataset each for cycling and boxing, revealing severe imbalance toward popular sports.Fitness-related datasets number 21, while racket, table, and bat-and-ball sports also receive substantial coverage.
  • Dataset Distribution by Application: Athlete-, coach-, and fan-focused data are robust, whereas referee-relevant datasets account for only 2% and data for researchers and industry remain scarce.Sports commentary generation, exercise prescription, and action recognition attract substantial attention.
  • Dataset Distribution by Modality: Video and text dominate dataset modalities, while audio is emerging and IMU sensor data and skeletal poses remain rare and concentrated in athlete motion analysis.The section argues that expanding less common modalities is important for cross-modal reasoning.
  • Dataset Distribution by Annotation Source: Expert annotation represents 6% of general datasets versus 41% of specialized sports understanding benchmarks, exposing a shortage of professional-grade labels.Manual annotation remains standard because it minimizes noise.
  • Dataset Distribution by Modeling Paradigm: Generative datasets comprise 78.5% of the landscape, compared with 21.5% for discriminative datasets supporting tasks such as commentary generation, tactical synthesis, action recognition, and game prediction.The distribution reflects a shift from categorical labeling toward descriptive and reasoning-intensive sports applications.

4 Discussion

The discussion identifies four fundamental challenges hindering robust, trustworthy real-world adoption of large models in sports and outlines four future directions toward practical use. Key concerns include dataset bias, fairness, privacy, latency, and long-video temporal reasoning, while proposed solutions emphasize trustworthy safeguards and more efficient temporal mechanisms.

  • Challenges: Existing sports large models face four fundamental challenges that hinder robust and trustworthy real-world adoption.The discussion frames these barriers as obstacles to practical deployment.
  • Bias, Fairness, and Privacy: Sports datasets overrepresent popular sports such as soccer and basketball, potentially causing unfair behavior and limiting generalization across sporting contexts.The passage links skewed representation to fairness and generalization concerns.
  • Real-Time and High-FPS Understanding: Latency-sensitive applications remain constrained by substantial Video LLM inference overhead, while hour-long broadcasts require difficult sustained temporal reasoning.These limitations affect live officiating, broadcast commentary, and long-video understanding.
  • Future Directions: Future research should strengthen trustworthiness through data balancing, cleaning, and sport-specific RLHF, while improving temporal processing with streaming inference and efficient KV-cache and attention mechanisms.These safeguards target bias, whereas the streaming mechanisms address sports’ low-latency and long-video demands.

5 Conclusion

The survey maps large models in sports across six stakeholder groups, analyzes relevant datasets, and highlights fundamental challenges. It consolidates these efforts into a framework and practical resource for advancing sports intelligence research and development.

  • The survey establishes a structured taxonomy spanning 6 stakeholder groups in large-model-driven sports intelligence.
  • It provides a deep analysis of relevant datasets and highlights fundamental challenges in the field.
  • By consolidating disparate research efforts, the survey offers a framework and practical resource for future research and development.

Limitations

The survey acknowledges that its coverage may lag behind rapidly evolving developments and may underrepresent research outside English-language publications, particularly from other regions.

  • Limitations: The survey may not timely reflect the latest developments in large models in sports because the field is rapidly evolving.This limitation applies to progress occurring before and after the survey.
  • Limitations: Its primarily English-language literature selection may limit coverage of domestic research in other regions and studies published in other languages.The selection follows standard protocols focused on English-language publications.

A Methodology for Literature Selection … A.3 Inclusion and Exclusion Criteria

The survey uses a PRISMA-aligned, bidirectional snowballing methodology, beginning with three expert-selected survey papers and iteratively expanding the literature until saturation. Studies are filtered by sports relevance, large-model use, publication type, technical rigor, and a January 2020–July 2025 time window.

  • A Methodology for Literature Selection: The literature-selection methodology combines systematic snowballing with PRISMA reporting standards to address the fragmented, rapidly evolving sports large-model landscape.The strategy uses both reference lists and citation networks.
  • A.1 Construction of the Start Set: Three comprehensive survey papers, identified through domain expertise, formed the initial start set for iterative snowballing.The seed papers were selected as the foundation because of their relevance and coverage.
  • A.2 Iterative Snowballing Procedure: The corpus expanded through iterative backward and forward snowballing, using reference lists and Google Scholar’s “Cited by” feature.A Boolean query was applied during the forward pass to manage scale and improve precision.
  • A.2 Iterative Snowballing Procedure: The forward-pass filter required “Sports” AND (“Large Language Model” OR “LLM” OR “GPT” OR “BERT” OR “T5”).This query targeted studies integrating large models within sports contexts.
  • A.2 Iterative Snowballing Procedure: The process captured research through July 2025, added eligible papers as new seeds, and continued until theoretical saturation produced no new relevant studies.The procedure repeated the search-and-expansion cycle until the filtered search yielded no additional relevant papers.
  • A.3 Inclusion and Exclusion Criteria: Included studies addressed sports tasks or sports-data analysis while using large models as core methodological components; studies relying solely on earlier methods were excluded.The topic filter focused the survey on large-model-era sports research.

A.4 Selection Results … B.6 Applications for the Sports Industry

The survey selected 241 core papers from approximately 2,200 candidate records and organizes large-model applications across athletes, educators, referees, fans, researchers, and sports organizations.

  • A.4 Selection Results: Approximately 2,200 candidate records were screened, yielding 241 core academic papers for the survey.The included works’ rapid growth and temporal distribution are presented in Figure 4.
  • B More Details on Large Model Applications in Sports: The application review systematically catalogs specific contributions and methodologies according to the taxonomy illustrated in Figure 2.It supplements the main text by focusing on individual research works rather than repeating task definitions, technology analyses, and evaluation metrics.
  • B.1 Applications for Athletes and Trainers: LLMs generate exercise prescriptions for health conditions and fitness goals, spanning weight management, resistance, jump, upper-body, and core training.They also support preventive advice, medical-information labeling, diagnostic imaging, data processing, clinical decisions, surgical planning, and outcome prediction.
  • B.2 Applications for Coaches and Educators: MLLMs support sports action spotting, recognition, technical-score quantification, tactical modeling, personalized exercise planning, and student mental-health support.Methods include keyframe sampling, contrastive pretraining, domain adaptation, event-sequence modeling, and graph representations.
  • B.3 Applications for Referees: Large models enhance refereeing fairness and transparency by pairing decisions with textual rationales, improving decision accuracy, consistency, and referee trust.X-VARS is an explainable Video Assistant Referee system fine-tuned on expert-annotated foul data.
  • B.4 Applications for Fans and Social Media: LLMs generate match commentary, social-media summaries, captions, personalized highlights, sports chatbots, co-viewing dialogue, and interactive search and retrieval.Applications use match events, tracking data, transcripts, role-play prompting, dialogue-state tracking, knowledge graphs, and multi-agent architectures.
  • B.5 Applications for Researchers: In sports science and medicine, LLMs assist with outlines, abstracts, grammar, and style, while raising concerns about accuracy, references, calculations, and originality.The literature specifically emphasizes caution regarding the reliability of generated content and citations.
  • B.6 Applications for the Sports Industry: Large models streamline sports-organization functions including financial strategy development, club-data management, facility-management dialogue, player-data analysis, and scouting-report structuring.Retrieval-augmented generation supports integration of diverse information sources.

C More Details on Datasets for Large Models in Sports

This section provides a comprehensive introduction to datasets for large models in sports, extending the main discussion presented in Section 3.

  • Scope: The section introduces datasets for large models in sports.It presents this discussion as a comprehensive introduction.
  • Relationship to Section 3: The dataset discussion extends the paper’s main treatment in Section 3.The section is explicitly framed as an extension of the earlier discussion.
  • Purpose: The section focuses specifically on providing further details about sports datasets used for large models.Its purpose is to supplement the main discussion with additional dataset coverage.

C.1 Task-Specific Datasets

Task-specific sports datasets cover five stakeholder groups but are unevenly developed across tasks, sports, and modalities. Research concentrates on tasks and sports with richer datasets, while several important areas remain underrepresented.

  • Task Coverage: Task-specific datasets span five sports stakeholder groups, with some tasks receiving substantially more research attention and possessing richer datasets than others.Sports commentary generation is comparatively well supported, whereas public opinion analysis and sports talent scouting lack open-source data.
  • Sport Coverage: Popular sports such as soccer, basketball, and badminton receive more attention and richer datasets, while track and field, aquatics, and esports are severely underrepresented.The underexplored sports are identified as valuable areas for further expansion and investigation.
  • Modality Coverage: Across modalities, video and text are the most common modalities represented in task-specific sports datasets.

C.2 Sports Understanding Datasets

This subsection surveys datasets supporting sports understanding with large models. It covers sports-specific datasets, general video-understanding datasets containing sports content, and other general-purpose datasets with sports-related data.

  • Overview: The subsection provides a detailed overview of datasets related to sports-understanding tasks for large models.It echoes Section 3.1 while expanding the dataset coverage.
  • Sports-Specific Datasets: Sports-understanding datasets designed specifically for large models are covered in §C.2.1.This is the first dataset category discussed.
  • General Video Datasets: General video-understanding datasets that include sports content are covered in §C.2.2.This is the second dataset category discussed.
  • General-Purpose Datasets: Other general-purpose datasets containing sports-related data are covered in §C.2.3.This is the third dataset category discussed.
  • Dataset Summary: Table 5 summarizes the first two dataset categories from multiple perspectives.The table provides a comprehensive summary of sports-specific and general video-understanding datasets.

C.2.1 Specialized Sports Understanding Datasets · C.2.2 General Video Understanding Datasets Featuring Sports · Other General Datasets Featuring Sports

The paper surveys specialized sports datasets, general video-understanding datasets containing sports, and other general datasets that incorporate sports content. These resources support evaluation of language, multimodal, perceptual, reasoning, action, audio-video, real-time, and domain-specific capabilities.

  • C.2.1 Specialized Sports Understanding Datasets: Specialized sports datasets evaluate large models’ sports understanding through contextual question answering, binary-choice questions, and multi-level multiple-choice assessment.QASports introduced a large-scale sports question-answering dataset; BIG-bench includes 986 binary-choice questions, while SportQA contains over 70,000 multiple-choice questions across three difficulty levels.
  • C.2.1 Specialized Sports Understanding Datasets: Multimodal sports datasets extend evaluation to sports video question answering and systematic video understanding across varied sports and difficulty levels.Sports-QA is described as the first dataset specifically designed for sports video question answering; SPORTU-video covers 7 sports, and Sports-3K-QA covers 49 sports.
  • C.2.2 General Video Understanding Datasets Featuring Sports: General video-understanding datasets featuring sports include broad multi-task resources and datasets focused on video description, subtitling, and viewpoint-specific understanding.InternVid, FIOVA, and VidText target description or subtitle generation, while Ego-Exo4D and EgoExoBench examine understanding from different viewpoints.
  • C.2.2 General Video Understanding Datasets Featuring Sports: Other general video datasets use sports content to assess long-video, fine-grained action, skill, and motion understanding.Examples include LVBench, MLVU, Neptune, LongVILA_sft, and VRBench for long videos, plus E.T. Bench, MotionBench, and ExAct for fine-grained capabilities.
  • C.2.2 General Video Understanding Datasets Featuring Sports: Some general video datasets featuring sports evaluate joint audio-video understanding, real-time processing, and the credibility of video-understanding models.The cited resources include video-SALMONN-2, WorldSense, HarmonySet, MAVERIX, OVO-Bench, RTV-Bench, and Trust-videoLLMs.
  • Other General Datasets Featuring Sports: Other general datasets incorporate sports scenes to test image perception, complex reasoning, and recognition of important figures.MDI-Benchmark collected 514 real images and 1,298 question-answer pairs, including sports-related questions for different age groups; MIP-GAF includes sports scenes.
  • Other General Datasets Featuring Sports: The paper also summarizes task-specific sports datasets spanning participants, industry applications, sports-understanding resources, and general video understanding with sports content.Tables 3–6 organize datasets by task categories, sports abbreviations, modalities, and source characteristics.
Loading 2608.14377v1…