ARC-Chapter: Structuring Hour-Long Videos into Navigable Chapters and Hierarchical Summaries

Motivation

• The framework targets hour-long videos, where multimodal long-range semantics, fine-grained annotations, and semantically appropriate boundary matching remain difficult.Existing approaches commonly focus on short videos, rely on limited annotations, or use metrics that inadequately match chapter granularity.Source: §1 · p. 1

Approach

• ARC-Chapter is trained at million-video scale and produces navigable chapters with hierarchical summaries spanning short titles, structural chapters, and timestamp-aligned descriptions.The structured output captures both coarse chapter identity and fine-grained temporally aligned content.Source: §1 · p. 1, Figure 1 · p. 2

Evaluation

• GRACE evaluates chapter boundaries using semantic granularity to better handle mismatches between predicted and reference segments.The metric is designed to address semantic misalignment in existing chaptering evaluation.Source: §1 · p. 2

Results

• ARC-Chapter establishes state-of-the-art performance on Chinese and English long-form chaptering benchmarks, with CIDEr improving from 100.9 to 186.6, F1 from 45.3 to 59.3, and SODA from 19.3 to 30.6.The model also shows consistent gains as training data and label density increase.Source: §1 · p. 2
• ARC-Chapter improves transferability to downstream video understanding tasks such as YouCook2 and ActivityNet Captions.The reported gains support the value of large-scale pretraining beyond chaptering benchmarks.Source: §1 · p. 2

Global Video Understanding

• Global video methods often treat videos as holistic units and struggle with the temporal dynamics and hierarchical structures of long-form content.This limitation is especially relevant when chapters must represent extended and nested content structure.Source: §2 · p. 2

Temporal Segmentation for Short Videos

• Temporal datasets enable fine-grained event localization and captioning, but most cover clips under several minutes rather than ultra-long videos.The lack of large-scale, long-duration datasets with fine-grained temporal annotations remains a bottleneck.Source: §2 · p. 3

Long-Form Video Structuring

• VidChapters-7M extends chaptering to millions of videos with annotated boundaries, reflecting long-term temporal reasoning in vlogs, podcasts, and meetings.It provides an important large-scale benchmark for real-world long-form structuring.Source: §2 · p. 3

Research Gap

• Existing chaptering models remain limited by narrow modalities, small-scale training, and coarse descriptions, motivating ARC-Chapter’s scalable multimodal design.The proposed framework is supported by a large-scale dataset with detailed chapter descriptions.Source: §2 · p. 3

3.1 Data Curation

• VidAtlas provides accurate chapter boundaries together with dense, multi-granularity descriptions for individual chapters and entire videos.The dataset is designed specifically for hierarchical chaptering and summarization.Source: §3.1 · p. 3
• Videos are selected using author-provided chapter markers containing timestamps and short titles, supplying human-verified temporal segmentation.Filtering retains videos between 2 minutes and 3 hours across diverse domains.Source: §3.1 · p. 4

3.2 Hierarchical Annotation

• The pipeline extracts timestamped ASR transcripts, visual captions, and OCR, then interleaves them into a unified chronological multimodal transcript for LLM reasoning.This avoids directly using multimodal LLMs for annotation while preserving information from audio and frames.Source: §3.2 · pp. 4–5
• The LLM reorganizes the transcript into chapters with comprehensive titles, abstracts, introductions, and precise boundaries, followed by verification against original timestamps.A second prompting stage produces a timestamped narrative description for the full video.Source: §3.2 · p. 5

3.3 Dataset Statistics

• VidAtlas contains more than 410k videos and 115k hours of content, averaging 5.5 chapters per video and covering 16 primary categories with over 100 subcategories.Its duration and topic diversity expose models to both rapid topic shifts and sustained thematic segments.Source: Figure 3 · p. 4, §3.3 · p. 5

4.1 Overall Framework

• ARC-Chapter integrates task prompts, sampled video frames, and timestamped ASR transcripts to generate short titles, structured chapters, or timestamped video descriptions.The vision encoder remains frozen while the language model is instruction-tuned for chaptering.Source: §4.1 · pp. 5–6, Figure 4 · p. 6

4.2 Training Strategy

• The model is trained with autoregressive next-token prediction and adaptive modality dropping across video-plus-ASR, video-only, and ASR-only inputs.This supports deployment when one modality is absent or noisy without requiring separate specialized models.Source: §4.2 · p. 7

4.3 Evaluation Metrics

• GRACE evaluates chaptering with many-to-one matching, combining temporal overlap and semantic similarity to tolerate differing annotation granularities.Groups are matched using DTW with chapter IoU as the matching criterion, while concatenated group captions are compared with BERTScore.Source: §4.3 · pp. 7–8, Figure 5 · p. 8

4.4 Reinforcement Learning with GRPO

• GRPO refines temporal localization with a reward that omits semantic BERTScore and sums temporal alignment over the optimal DTW matching.The reinforcement-learning phase uses video-only inputs and starts from the best supervised fine-tuned model.Source: §4.4 · p. 8
• ARC-Chapter achieves the best performance across all metrics and video durations on VidChapters7M-test, while Table 2 reports superior performance from integrating speech and video.The supplied table descriptions establish the comparisons but do not provide the underlying cell values.Source: Table 1 · p. 9, Table 2 · p. 9

• The experiments first introduce evaluation benchmarks, then report main results and detailed ablation studies.Source: §5 · p. 9

• VidChapters7M-test contains 8.2k English samples and is evaluated with ASR transcripts, while VidAtlas-test covers over 1.5k Chinese videos with ASR and original-video inputs.Source: §5.1 · pp. 9–10
• The evaluation benchmarks target both temporal boundary precision and the semantic relevance of generated chapter titles and descriptions.Source: §5.1 · p. 9
• ARC-Chapter is compared across three benchmarks covering different languages, scales, and data modalities.Source: §5.1 · pp. 9–10

• 54.5 F1, 76.7 tIoU, 23.5 SODA, and 144.0 CIDEr establish ARC-Chapter as state of the art on VidChapters7M-test, improving over Chapter-Llama by +9.2 F1, +4.9 tIoU, and +6.0 SODA.The performance advantage increases with video duration, particularly for 30–60-minute videos.Source: §5.2 · p. 10
• 66.2 F1, 84.0 tIoU, 30.2 SODA, 141.5 CIDEr, and 34.1 GRACE make ARCChapter-vidasr state of the art on VidAtlas, surpassing Gemini-2.5-Pro by +17.5 F1 and +16.7 SODA.ARC-Chapter also leads in ASR-only and video-only settings, with F1 scores of 58.8 and 57.6, respectively.Source: §5.2 · p. 10

• 37.9/12.5 F1/SODA on YouCook2 surpasses the previous best of 33.5/7.9 for event segmentation.The model establishes state-of-the-art performance across YouCook2 and ActivityNet Captions after pre-training followed by fine-tuning.Source: §5.3 · p. 10
• The transferability evaluation uses visual inputs without ASR and reports overall dataset ranking as the mean rank across M, S, C, and F1 metrics.Table 4 covers YouCook2 and ActivityNet Captions, with some ActivityNet results sourced from [14] and [46].Source: Table 4 · p. 11

5.4.1 Scaling Property

• Training-data performance improves positively across F1, tIoU, SODA, and CIDEr for all modalities, while the full Video+ASR model consistently performs best.ARC-Chapter remains strong with 20% of the training data and continues improving as the corpus grows.Source: §5.4.1 · p. 11, Figure 6 · p. 12

5.4.2 Hierarchical Annotations

• Hierarchical training enables strong title, abstract, and introduction generation on both benchmarks, particularly with combined video and ASR inputs.The model learns complex structured outputs from Short Title to Structural Info, supporting semantic understanding.Source: §5.4.2 · p. 11, Table 5 · p. 12
• Only a small segmentation gap separates Short Title from Structural Info: multimodal VidChapter-sml300 performance declines from 62.4 to 61.4 F1 and from 81.6 to 80.6 tIoU.The results indicate that detailed multi-part generation can be performed without materially compromising chapter boundary accuracy.Source: §5.4.2 · p. 11, Table 5 · p. 12

5.4.3 Performance with GRPO

• +0.8 F1 and +0.7 tIoU on VidAtlas video input show that GRPO improves temporal segmentation over SFT.The reinforcement-learning stage is evaluated by comparing SFT-base with the GRPO-enhanced model.Source: §5.4.3 · p. 12, Table 6 · p. 13
• +1.5 F1 and +1.1 tIoU for Video+ASR on VidChapter7M-test demonstrate cross-modal gains despite GRPO training using video only.This pattern supports refinement of a modality-agnostic temporal representation.Source: §5.4.3 · p. 12
• +1.1 CIDEr for video input on VidChapters7M illustrates that GRPO improves temporal precision without sacrificing semantic quality.SODA and GRACE remain stable or improve slightly, consistent with KL-regularized optimization avoiding policy degradation.Source: §5.4.3 · pp. 12–13, Table 6 · p. 13

5.5 Qualitative Visualization

• ARC-Chapter generates accurate, coherent, and hierarchically structured outputs across English and Chinese qualitative examples.The visualizations cover short titles, timestamped descriptions, and structural chapters.Source: §5.5 · p. 13
• On a dense English finance video, short titles segment logical thematic units while structural chapters provide complementary titles, abstracts, and introductions.The outputs capture both chapter boundaries and layered content structure.Source: §5.5 · p. 13
• On a Chinese video about a similar topic, the model produces precise short titles, fluent descriptions, and contextually appropriate structural chapters.The comparable quality supports multilingual chaptering and summarization.Source: §5.5 · p. 13
• The qualitative examples portray ARC-Chapter as a versatile video-understanding model for rich, structured, multilingual summaries useful to end-users.Source: §5.5 · p. 13

6 Conclusion

• ARC-Chapter combines a large-scale dataset of millions of long video chapters with a semi-automatic annotation pipeline for structuring long-form videos.The framework targets semantically coherent chapters and hierarchical summaries.Source: §6 · p. 13
• GRACE provides a granularity-robust assessment of chapter boundaries that addresses limitations in existing evaluation methods.Source: §6 · p. 13
• ARC-Chapter achieves superior performance across multiple benchmarks, video durations, and languages, demonstrating effectiveness and generalizability.Source: §6 · p. 13
• The framework has strong potential to facilitate efficient content navigation, retrieval, and understanding as long-form video continues to grow.Source: §6 · p. 13