Source-linked AI summary
YouTube UGC Dataset for Video Compression Research
Yilin Wang, Sasi Inguva, Balu Adsumilli
TL;DR
UGC often violates pristine-reference assumptions, leaving compression and quality assessment with limited public datasets. This paper constructs and samples a large YouTube UGC dataset across content and resolution dimensions, then discusses reference-metric shortcomings and evaluates no-reference metrics. The resulting resource covers 1,500 standardized 20-second clips and supports research on practical UGC compression and quality assessment.
Problem
UGC commonly contains non-pristine originals, while traditional reference-based metrics assume perfect originals and few public UGC datasets support research.
Method
The paper samples millions of Creative Commons YouTube videos using encoding-derived features across 15 categories and resolution subgroups, creating standardized raw clips.
Results
The final dataset contains 1,500 20-second clips in raw YUV 4:2:0 format with constant framerate.
Takeaways & Limitations
The dataset is intended to encourage research addressing practical requirements of UGC video compression and quality assessment.
Abstract
from arXiv · showhide
Non-professional video, commonly known as User Generated Content (UGC) has become very popular in today's video sharing applications. However, traditional metrics used in compression and quality assessment, like BD-Rate and PSNR, are designed for pristine originals. Thus, their accuracy drops significantly when being applied on non-pristine originals (the majority of UGC). Understanding difficulties for compression and quality assessment in the scenario of UGC is important, but there are few public UGC datasets available for research. This paper introduces a large scale UGC dataset (1500 20 sec video clips) sampled from millions of YouTube videos. The dataset covers popular categories like Gaming, Sports, and new features like High Dynamic Range (HDR). Besides a novel sampling method based on features extracted from encoding, challenges for UGC compression and quality evaluation are also discussed. Shortcomings of traditional reference-based metrics on UGC are addressed. We demonstrate a promising way to evaluate UGC quality by no-reference objective quality metrics, and evaluate the current dataset with three no-reference metrics (Noise, Banding, and SLEEQ).
I. INTRODUCTION
UGC complicates video compression and quality assessment because originals often contain artifacts, making pristine-reference assumptions unreliable. The paper therefore develops a large-scale dataset to support more realistic research and evaluates no-reference alternatives.
- Video bitrate reduction controls bandwidth but can damage perceptual quality as streaming expectations rise.
- Reference-based metrics assume pristine originals, an assumption that often fails for User Generated Content.
- Original artifacts create a two-sided compression problem: encoders must represent them while also accounting for their effect on encoding efficiency.
- Reference-based metrics cannot fairly assess quality changes from pre- or post-processing when the original is imperfect.
- The paper analyzes YouTube content, builds a statistically representative UGC dataset, proposes encoding-log-based sampling, and evaluates Noise, Banding, and SLEEQ.
II. RELATED WORK
Existing UGC datasets support quality assessment or provide extracted features, but compression research needs accessible raw video data. Prior datasets vary substantially in scale, format, resolution, and subjective annotation.
- YouTube-8M and AVA provide extracted features rather than raw pixels, limiting their usefulness for compression research.
- Xiph.org Video Test Media offers around 120 clips spanning pristine and UGC samples, resolutions from SD to 4K, and categories including movies and gaming.
- LIVE datasets contain 150 distorted versions evaluated by 35 to 55 human subjects for subjective quality assessment.
- VideoSet contains 220 five-second clips from 11 videos and reports three JND scores collected from more than 30 subjects.
- The Crowdsourced Video Quality Dataset contains 585 ten-second clips from 80 inexpert videographers across 18 resolutions, with crowdsourced opinions.
- KoNViD-1k contains 1,200 landscape clips above 960 × 540, sampled from 150K videos and paired with crowdsourced subjective scores.
III. YOUTUBE UGC DATASET OVERVIEW
The dataset samples YouTube UGC across content categories and resolution levels to represent varied video characteristics. It contains 1,500 standardized 20-second raw clips for research.
- The dataset samples 1.5 million Creative Commons YouTube videos across 15 categories, including Gaming, HDR, Sports, Vertical Video, Vlog, and VR.
- Content categories expose differing visual characteristics, such as fast motion in Gaming and still backgrounds in Lyric videos, relevant to compression optimization.
- Resolution is treated as an independent dimension, with 360P, 480P, 720P, and 1080P broadly represented and 4K included for six categories.The six categories with 4K are HDR, Gaming, Sports, Vertical Video, Vlog, and VR.
- The final dataset contains 1,500 20-second clips in raw YUV 4:2:0 format with constant framerate.
IV. DATASET SAMPLING
The dataset samples representative 20-second YouTube clips across categories, resolutions, and four encoding-derived complexity spaces. Its sampling scheme targets broad feature coverage while avoiding duplicate source videos.
- Feature representation: Each candidate clip is represented by spatial, color, temporal, and chunk-variation complexity features extracted from encoding logs.Features are collected over 20-second windows stepped by 1 second through each video.
- Feature representation: Temporal complexity uses P-frame bits normalized by I-frame bits to reduce the influence of spatial complexity.Frequent changes in the temporal row-sum map indicate faster motion.
- Feature representation: Chunk variation is the frame-size-normalized standard deviation of compressed bitrates across 1-second chunks, capturing within-video quality variation.Multiple scene changes produce distinct regions in the row-sum map.
- Sampling procedure: The sampling procedure normalizes each feature, divides its range into three bins, permutes non-empty bins, and selects clips separated by a Euclidean distance threshold of 0.3.It also prevents selecting multiple clips from the same original video.
- Sampling outcome: 89% average coverage is achieved across pairwise feature-space grids, while the sampled complexity distributions are less spiky than the initial distributions.The dataset retains 15 to 25 selected clips from each category–resolution subgroup after manual removal of mislabeled clips.
V. CHALLENGES ON UGC QUALITY ASSESSMENT
Traditional reference-based scores can be misleading for non-pristine UGC because original artifacts distort comparisons. No-reference metrics offer a possible alternative for assessing compression effects and revealing category-specific quality issues.
- 29.02 PSNR, 0.86 SSIM, and 58.15 VMAF can accompany visually similar original and compressed clips.The examples use H.264 compression with CRF 32.
- 30.28 PSNR, 0.63 SSIM, and 31.43 VMAF remain misleading when the compressed clip has less noise than the original.Both versions contain artifacts, but the compressed version in Fig. 6 has fewer noise artifacts.
- Original artifacts cause low reference-quality scores because PSNR, SSIM, and VMAF do not correctly account for non-pristine references.
- No-reference evaluation uses Noise, Banding, and SLEEQ, with the first two targeting specific artifacts and SLEEQ measuring compression artifacts in natural scenes.The paper states that all three metrics correlate well with human ratings.
- Animation videos show more banding artifacts, while nature scenes in Vlog tend to contain more artifacts than other categories.
VI. CONCLUSION
The paper presents a large, representative YouTube UGC dataset and a sampling scheme based on features extracted from millions of videos. It examines UGC quality-assessment difficulties while leaving compression degradation with non-pristine references as an open question.
- The dataset is designed to highly represent videos uploaded to YouTube.
- A novel sampling scheme extracts features from millions of video samples and supports analysis across 15 categories.
- Evaluating compression-caused quality degradation with non-pristine references remains an important open question.