Source-linked AI summary
Title Generation for User Generated Videos
Kuo-Hao Zeng, Tseng-Hung Chen, Juan Carlos Niebles, Min Sun
TL;DR
Video title generation is less addressed than captioning and must identify a salient event within a long user-generated video. The paper adapts captioners with highlight sensitivity and sentence augmentation; its methods improve VTW title prediction, and sentence augmentation also outperforms the S2VT baseline on M-VAD.
Problem
Video title generation is less addressed than captioning and must compactly describe a salient event in a long user-generated video rather than the video as a whole.
Source: §1 · p. 1, §3 · p. 5, §4.2 · p. 9Method
The paper adapts video captioners with highlight-sensitive training and sentence augmentation using additional sentence-only examples.
Source: §1 · p. 3, §4.2 · p. 9Results
The methods consistently improve VTW title prediction, with both methods together achieving the best performance; on M-VAD, sentence augmentation reaches 7.1% METEOR versus the S2VT baseline’s 6.7%.
Source: §6 · pp. 14–15Takeaways & Limitations
VTW is complementary to movie-description datasets for future study of video captioning and title generation.
Source: §3.2 · p. 8
Abstract
from arXiv · showhide
A great video title describes the most salient event compactly and captures the viewer's attention. In contrast, video captioning tends to generate sentences that describe the video as a whole. Although generating a video title automatically is a very useful task, it is much less addressed than video captioning. We address video title generation for the first time by proposing two methods that extend state-of-the-art video captioners to this new task. First, we make video captioners highlight sensitive by priming them with a highlight detector. Our framework allows for jointly training a model for title generation and video highlight localization. Second, we induce high sentence diversity in video captioners, so that the generated titles are also diverse and catchy. This means that a large number of sentences might be required to learn the sentence structure of titles. Hence, we propose a novel sentence augmentation method to train a captioner with additional sentence-only examples that come without corresponding videos. We collected a large-scale Video Titles in the Wild (VTW) dataset of 18100 automatically crawled user-generated videos and titles. On VTW, our methods consistently improve title prediction accuracy, and achieve the best performance in both automatic and human evaluation. Finally, our sentence augmentation method also outperforms the baselines on the M-VAD dataset.
1 Introduction
Video title generation targets a compact, attention-catching description of a video's most salient event, unlike captioning, which describes the video as a whole. The paper addresses highlight alignment and title diversity with highlight-sensitive captioning and sentence augmentation, and introduces the VTW dataset.
1 Introduction
- Video titles compactly describe a video's most salient event and capture attention, whereas captions describe the video as a whole.The example contrasts a short highlight with captions covering a longer sequence.Source: §1 · p. 1, Figure 1 · p. 2
- Untrimmed user-generated videos make it difficult to align title words with the relevant highlight, while title sentences are highly diverse.In VTW, each word appears in only two sentences on average.Source: §1 · p. 2
- The highlight-sensitive captioner combines a highlight detector with captioning and jointly learns to generate titles and locate highlights.Training highlights can also improve the detector.Source: §1 · p. 2
- Sentence augmentation adds sentence-only examples, associated with a shared dummy video observation, so captioners can train on unpaired text.The approach is intended to leverage additional sentences available on the web.Source: §1 · pp. 2–3
- VTW contains 18,100 automatically crawled user-generated videos and titles; the methods improve title prediction, with the combined approach reaching 6.2% METEOR and 25.4% CIDEr.The paper also reports sentence augmentation improving METEOR on M-VAD.Source: §1 · p. 3
2 Related Work
Related work spans staged, template-based video description and learned CNN–RNN captioning, alongside highlight detection and video-description datasets. The paper positions title generation as distinct because it requires selecting a salient event in longer videos.
Video Captioning
- Early video-captioning systems classified visual content, then combined classifier scores with a language model and templates to generate sentences.Their engineered components often limited sentence diversity.Source: §2 · pp. 3–4
- Learned CNN–RNN approaches map visual features to sentences, with later video methods modeling temporal structure or attending to observations for each word.The proposed highlight-sensitive method is characterized as hard attention over a video segment.Source: §2 · p. 4
Video Highlight Detection
- Prior highlight detection includes sports-focused methods, generic-personal-video methods, and unsupervised summarization, while this paper combines explicit highlight detection with sentence generation.Song et al. use video titles to summarize videos but require additional images retrieved by title search.Source: §2 · pp. 4–5
Video Captioning Datasets
- Existing video-description datasets include short clips and movie-derived descriptions, whereas VTW is collected automatically from user-generated videos and is complementary to movie datasets.The cited comparison says short-clip datasets are unsuitable for title-generation evaluation because salient-event selection is not critical there.Source: §2 · p. 5
3 Video Title Generation
Video title generation aims to describe the most salient event in a user-generated video, and the section introduces VTW as a dataset for studying the task. Its long videos, largely unknown highlights, and diverse text distinguish it from existing video-description datasets.
3 Video Title Generation
- Video title generation extends captioning from describing a short clip to describing the most salient event in a typical one-minute user-generated video.The section presents VTW and its properties before introducing the proposed methods.Source: §3 · p. 5
3.1 Collection of Curated UGVs
- VTW comprises 18,100 open-domain videos averaging 1.5 minutes, with editor-produced titles and longer descriptions that may not match the salient event or visible content.Titles serve as ground truth; the collection was automatically gathered without further curation.Source: §3.1 · p. 6
- VTW videos are five to ten times longer than prior video-captioning clips, and manually labeled highlights have a median duration of about 3.3 seconds.The temporal location and extent of highlights are unknown for most videos, making word-to-video alignment challenging.Source: §3.1 · p. 6
3.2 Dataset Comparison
- The VTW dataset has the longest duration among compared datasets at 213.2 hours, with each video about 10 times longer than each clip in [5].Its total duration is 2.5 times longer than [5].Source: §3.2 · p. 6, Table 1 · p. 7
- VTW averages about two sentences per word, versus 5.3 in MSVD, indicating more diverse sentences and a need for more training text.The datasets have similar distributions of nouns, verbs, adjectives, and adverbs, but VTW vocabulary differs from movie-description datasets because it covers user-generated videos.Source: §3.2 · pp. 7–8, Table 2 · p. 7
4 From Caption to Title
The paper extends video captioning to title generation by addressing two challenges: aligning titles with unknown highlights and modeling diverse sentence structures. It introduces highlight-sensitive captioning and sentence augmentation to handle these challenges.
4.1 Video Captioning
- Captioners must model both word–video grounding and word-sequence structure; title generation adds unknown highlights and diverse sentences as challenges.The paper presents methods for improving both relations.Source: §4.1 · pp. 8–9
4.2 Highlight Sensitive Captioning
- For long untrimmed videos, the method alternates between training the captioner and selecting title-relevant highlights, initialized by a detector trained on labeled examples.Updated highlights also retrain the detector and captioner.Source: §4.2 · pp. 9–10
4.3 Sentence Augmentation
- Sentence augmentation trains captioners on video–sentence pairs and sentence-only examples by assigning the latter dummy video observations.SA uses a dummy observation that acts as a trainable bias; S2VT uses an all-zeros dummy vector, which performed better for S2VT in the experiments.Source: §4.3 · pp. 10–11
5 Experiments
The experiments evaluate highlight-sensitive captioning and sentence augmentation on VTW, with selected results also tested on M-VAD. The combined highlight and Web Aug. method performs best on VTW, while sentence augmentation improves over the S2VT baseline on M-VAD.
5 Experiments
- VTW experiments use an 80/10/10 train-validation-test split, with 14,100 training pairs and manually labeled highlight moments for detector training and evaluation.The initial bidirectional RNN detector predicts clip highlightness; captioners use eight consecutive highlight clips, and the detector and captioner are updated iteratively.Source: §5 · p. 11
- Video features combine VGG frame appearance features with C3D motion features, while evaluation uses BLEU, METEOR, CIDEr, and human title preferences.The authors motivate CIDEr because it rewards correctly predicted rare n-grams, which they consider important for titles.Source: §5 · pp. 11–12
5.1 Baseline Methods
- The compared variants include Vanilla, highlight-sensitive models, description-based augmentation, Web Aug., and HL+Web Aug.; descriptions paired with their original videos are treated as a risky assumption.Web Aug. retrieves similar sentences as augmentation, and HL+Web Aug. initializes from the trained Web Aug. model before highlight-sensitive finetuning.Source: §5.1 · p. 12
5.2 Results
- On VTW, HL+Web Aug. achieves the best accuracy across all measures, and human judges rate 59.5% of its S2VT titles on par with or better than Vanilla.For S2VT, HL improves over HL-1 on B@1-4, METEOR 6.2%, and CIDEr 24.9%; on SA, HL also improves over Vanilla and HL-1 on METEOR 5.6% and CIDEr 24.9%.Source: Table 3 · p. 13, §5.2 · p. 13
- The highlight-sensitive method raises the updated detector's testing performance to 58.3% mAP from the initial detector's 54.2% mAP.The authors also report that Vanilla-GT-HL does not outperform Vanilla, while an HL-0 model trained without highlight supervision is only slightly inferior to HL on S2VT.Source: §5.2 · p. 13
- On M-VAD, S2VT+Web Aug. reaches METEOR 7.1%, versus 6.6% for the S2VT baseline and 6.7% reported in.The paper presents this as evidence that sentence augmentation can improve video captioning accuracy across datasets.Source: §5.2 · p. 14
6 Conclusion
The conclusion frames video title generation as a more challenging task than video captioning and reports that the proposed methods improve title prediction on VTW. The best VTW performance combines highlight-sensitive captioning with sentence augmentation, and sentence augmentation also outperforms the S2VT baseline on M-VAD.
6 Conclusion
- The paper introduces video title generation as a more challenging task than video captioning.Its methods extend state-of-the-art video captioners to generate titles.Source: §6 · p. 14
- On VTW, the proposed methods consistently improve title prediction accuracy, with the best performance from combining highlight-sensitive captioning and sentence augmentation.Source: §6 · pp. 14–15
- On M-VAD, sentence augmentation reaches METEOR 7.1%, outperforming the S2VT baseline at 6.7% in.Source: §6 · p. 15