Source-linked AI summary
Understanding the Characteristics of Internet Short Video Sharing: YouTube as a Case Study
Xu Cheng, Cameron Dale, Jiangchuan Liu
TL;DR
The paper asks how YouTube’s fast-growing short-video service differs from traditional video repositories and how its social structure can be characterized. Using three months of crawled data, it measures video, usage, growth, and social-network properties. It finds distinct YouTube characteristics and a small-world related-video network, with implications for delivery strategies.
Problem
Understanding YouTube’s features matters because the service affects network traffic engineering and sustainable development of video-sharing services.
Method
The authors conduct a systematic measurement study using 27 datasets totaling 2,676,388 videos crawled over three months, including analysis of related-video links.
Results
YouTube differs from traditional video repositories in length distribution, access pattern, and growth trend, while related-video networks exhibit small-world characteristics.
Takeaways & Limitations
The observed characteristics create challenges and opportunities for optimizing short-video sharing, including caching or peer-to-peer delivery strategies.
Abstract
from arXiv · showhide
Established in 2005, YouTube has become the most successful Internet site providing a new generation of short video sharing service. Today, YouTube alone comprises approximately 20% of all HTTP traffic, or nearly 10% of all traffic on the Internet. Understanding the features of YouTube and similar video sharing sites is thus crucial to their sustainable development and to network traffic engineering. In this paper, using traces crawled in a 3-month period, we present an in-depth and systematic measurement study on the characteristics of YouTube videos. We find that YouTube videos have noticeably different statistics compared to traditional streaming videos, ranging from length and access pattern, to their active life span, ratings, and comments. The series of datasets also allows us to identify the growth trend of this fast evolving Internet site in various aspects, which has seldom been explored before. We also look closely at the social networking aspect of YouTube, as this is a key driving force toward its success. In particular, we find that the links to related videos generated by uploaders' choices form a small-world network. This suggests that the videos have strong correlations with each other, and creates opportunities for developing novel caching or peer-to-peer distribution schemes to efficiently deliver videos to end users.
I. INTRODUCTION
YouTube’s rapid growth reflects both an integrated platform for sharing and discussing videos and a social network linking users and content. This study measures the resulting service characteristics and their implications for video delivery.
- Motivation: YouTube grew into a major Internet service, creating significant traffic-engineering and scalability concerns.The site was described as the fourth most accessed site and as having a substantial impact on Internet traffic distribution.
- Study scope: The paper presents a systematic measurement study based on 27 datasets totaling 2,676,388 videos crawled over three months.The datasets support analysis of video statistics, newer features such as ratings and comments, and growth trends.
- Social network: Related-video links selected by uploaders form a small-world network, suggesting strong video correlations and opportunities for caching or peer-to-peer delivery.The paper connects this network structure to possible strategies for efficiently delivering videos to end users.
- Motivation: Its integrated platform makes uploading, sharing, watching, rating, and commenting on videos easier than with earlier services.Earlier videos were standalone units with cumbersome management and little content review; YouTube addressed these limitations.
- Motivation: Social connections among users and videos substantially contributed to the success of YouTube and similar sites.The paper treats this social network as a central characteristic of the new video-sharing services.
B. Workload Measurement of Traditional Media Servers
The paper positions YouTube as a workload distinct from traditional media-server and video-on-demand services. Its measurement approach combines repeated repository crawls with metadata and web-page collection, while YouTube’s delivery model uses ordinary HTTP downloads.
- Related work: Traditional media-server research has examined video popularity, access locality, and differences among media and streaming workloads.The authors use this prior work as the basis for comparing YouTube’s workload characteristics.
- Related work: MSN Video was a similar measurement target, but it had fewer, longer videos and lacked related-video listings and user information.These differences distinguish MSN Video’s traditional service model from YouTube’s social video-sharing environment.
- Measurement approach: The study crawled YouTube for three months using both the YouTube API and scraped video webpages to obtain repository snapshots and changing trends.The resulting data focus on access patterns and social networks present in YouTube.
- Delivery model: YouTube videos are downloaded over normal HTTP at the maximum achievable rate rather than streamed at the playback rate.From the server’s perspective, user interactivity is limited, apart from possibly stopping the download.
- Video metadata: Each video’s metadata includes uploader, upload date, category, length, views, ratings, comments, and related videos.Related videos are links based on similar titles, descriptions, or tags and are selected by the uploader.
B. YouTube Crawler
The crawler models YouTube as a directed graph and explores it through related-video links, while repeated crawls and targeted retrievals support workload, growth, and user analyses. The resulting repository sample is large and generally active.
- B. YouTube Crawler: The crawler represents each video as a graph node and each related-video link as a directed edge.It uses breadth-first search, starting from an initial queue of video IDs and adding previously unseen related videos.
- B. YouTube Crawler: The first crawl began with 189 featured, viewed, rated, or discussed videos and found approximately 750 thousand videos in about five days.The crawl reached more than four depths, although the fifth depth was not completed.
- B. YouTube Crawler: Subsequent crawls ran every two to three days from roughly 200 to 300 starting videos and found 80 thousand videos on average in less than 10 hours.These repeated crawls provided successive repository snapshots.
- B. YouTube Crawler: Seven weekly datasets updated view counts for relatively new videos from March 5 through April 16, 2007, enabling growth-trend analysis.The targeted updates covered videos uploaded after February 15, 2007.
- B. YouTube Crawler: The crawler separately collected file sizes, embedded bit rates, and user upload and friend counts.User information was retrieved for more than 1 million users, while file size came from download responses.
- B. YouTube Crawler: The final collection contained 27 datasets and 2,676,388 videos, representing a significant portion of YouTube’s estimated repository.Most sampled videos were reachable from the homepage in fewer than 10 clicks and were therefore treated as generally active and representative.
- B. YouTube Crawler: Static attributes were measured once, while dynamic views, ratings, and comments were treated as fixed within each crawl and updated later for growth analysis.This distinction separates repository metadata from measurements that change across datasets.
A. Video Category
YouTube’s video categories are highly skewed, with Music the largest category, while video lengths and bit-rates show distinct concentration patterns across the repository.
- Category distribution: Music is the largest category at about 22.9%, followed by Entertainment at about 17.8% and Comedy at about 12.1%.
- Video length: 97.8% of videos are at most 600 seconds long, and 99.1% are at most 700 seconds long.The 10-minute upload limit contributes to this concentration, although some longer videos exist.
- Video length: The length distribution has three peaks: within one minute, between three and four minutes, and near the 10-minute maximum.The first peak contains more than 20% of videos, while the second contains about 16.7%; Music contributes substantially to the second peak.
- Category-specific length: Music videos peak strongly between three and four minutes, whereas Comedy and Sports videos contain more videos within two minutes.Entertainment videos show a similar but smaller three-to-four-minute peak.
C. File Size and Bit-rate
YouTube videos are generally small and use moderate bit-rates, while the crawled upload data shows a steep post-launch growth trend with sampling bias for recent videos.
- File Size: 98.8% of crawled videos are less than 30MB, with an average file size of about 8.4 MBytes.Extrapolating to over 42.5 million videos gives more than 357 terabytes of total storage.
- Bit-rate: 87.6% of crawled videos contain FLV metadata specifying their bit-rate at the beginning of the file, indicating CBR encoding.The remaining videos likely use VBR encoding, so their average bit-rate is calculated from file size and length.
- Bit-rate: Most videos have a bit-rate around 330 kbps, with additional peaks around 285 kbps and 200 kbps.The three peaks suggest a moderate bit-rate balancing video quality and bandwidth.
- Date Added: After six months, the number of uploaded videos increases steeply, and the trend is fitted with a power-law curve.The earliest crawled video was uploaded on April 27th, 2005, although early videos were observed only if still popular or linked.
- Date Added: The apparent steep linear decrease in videos added from March 2007 reflects crawler visibility rather than a demonstrated decline in YouTube’s upload rate.Recently uploaded videos were less likely to be popular or linked from related-video lists, though they could become discoverable later.
E. Views, Ratings – User Access Pattern
Views, ratings, and comments exhibit skewed access distributions, but views depart from Zipf’s law through a sharp tail drop-off; Weibull and Gamma fit better than Zipf.
- Views: Video views do not follow a Zipf distribution: after the 2 × 10^3-ranked video, the tail decreases tremendously.The beginning is linear on a log-log scale, while the tail contains fewer less-popular videos than Zipf’s law predicts.
- Views: Weibull and Gamma distributions fit the skewed view distribution better than Zipf because they capture the tail drop-off.The analysis uses a relatively static April 3rd, 2007 dataset containing more than 100 thousand videos.
- Ratings and Comments: Ratings and comments have the same distribution as each other and resemble the view distribution, but their tails drop less quickly.Each distribution is ranked by the corresponding number of ratings or comments.
F. Growth Trend of Number of Views and Active Life Span
The paper models video popularity growth with a power law and uses its growth factor to define active life span, finding that most videos slow over time and usually remain active briefly.
- Growth Trend: The growth trend factor p is the exponent in the power-law model, distinguishing increasing, near-constant, and slowing popularity growth.Values above 1 indicate increasing growth, values near 1 relatively constant growth, and values below 1 slowing growth.
- Growth Trend: Over 70% of videos have a growth trend factor p less than 1, indicating that most videos grow in popularity more slowly as time passes.The distribution is estimated by modeling approximately 43 thousand videos with the power-law growth model.
- Active Life Span: Active life span depends on p and the number of weeks a video has been on YouTube, but not on its starting view count.The definition uses a relative week-to-week growth threshold rather than an absolute view-count threshold.
- Active Life Span: For approximately 30 thousand videos with p less than 1 and t = 10%, a Pareto distribution fits active life span well with parameter k = 1.06.Fits across multiple t values produce the same k, differing only in line location.
- Active Life Span: Most videos have a short active life span, after which progressively fewer people access them.Active life span provides an indirect view of temporal locality because server logs were unavailable for accurate direct measurement.
- Implications: The active-life model can support predictors that help proxies or servers decide when to drop videos from cache.This application is presented as an implication for web caching and server storage.
V. THE SOCIAL NETWORK IN YOUTUBE
YouTube videos and users form an interconnected social network rather than independent media objects, making the network’s structure important to analyze.
- Social Network: YouTube includes communities, groups, video and channel statistics, ratings, comments, and other social features linking users and videos.These features make videos no longer independent from one another, unlike traditional media services.
- Social Network: The paper therefore examines the social network among YouTube users and videos as a distinctive characteristic of video-sharing sites.The analysis treats this network as a unique aspect compared with traditional media services.
A. User Friends and Upload
The paper examines YouTube users and their uploads, finding that user connectivity and upload activity follow skewed distributions and that many users have no friends.
- User data: Over 1 million users were analyzed for numbers of friends and uploaded videos.The data came from a May 28, 2007 crawl using users found in earlier crawls.
- User friends: 58% of users have no friends.The authors suggest YouTube’s relative youth may partly explain the limited connections.
- User friends: Friend counts are much closer to linear on a log-log rank plot than earlier plots, though three distributions provide the best fit.The result is based on data from more than 1 million users.
- User uploads: The rank distribution of uploaded-video counts is very similar to those of views and friends.The corresponding plot is omitted for brevity.
C. The Small-World in YouTube
Using related-video links to construct video graphs, the paper finds strong small-world characteristics: high clustering and short path lengths approaching random-graph values in larger datasets.
- Graph construction: Related links formed directed video graphs, with videos lacking incoming or outgoing links removed.A combined graph was also created, but incomplete crawling prevented the resulting graphs from being strongly connected.
- Graph construction: Random comparison graphs matched the crawled strongly connected components in node count and average degree, with maximum out-degree limited to 20.No comparable random graph was generated for the combined dataset because it was too large.
- Clustering: The clustering coefficient is high relative to random graphs and drops noticeably for the largest datasets.The decrease indicates inverse dependence on graph size, a pattern reported for some small-world networks.
- Path length: Once datasets reach a few thousand nodes, the diameter decreases as small-world structure becomes evident.For the largest datasets, average diameter is only slightly larger than that of a random graph.
- Small-world result: YouTube’s related-video network combines very large clustering coefficients with characteristic path lengths approaching those of random graphs.The links are generated using user-created tags, titles, and descriptions.
- Comparison: The YouTube video network may be more closely grouped than the Web, whose characteristic path length is reported as 18.59.The passage notes that the difference may reflect the Web’s much larger node count or a closer YouTube network.
A. Implications on Proxy Caching and Storage Management
The measurements motivate caching and storage strategies tailored to YouTube’s related-video structure and evolving video popularity, including prefix caching, lifespan prediction, and neighbor prefetching.
- Proxy caching: Prefix caching is presented as more practical than full-object or segment caching for YouTube.The proposed example caches a 5-second initial clip, approximately 200KB, for each video.
- Cache management: A Pareto-distributed active lifespan supports predictors that identify videos whose popularity has ended and enable cached-prefix replacement.The predictor is intended to address when cached space should be released as the repository evolves.
- Storage management: Active-lifespan prediction can also support hierarchical server storage by moving videos past their active lifespan to slower, cheaper media.The motivation includes 65,000 new videos introduced daily, which threatens server storage capacity.
- Related-video prefetching: YouTube’s small-world related-video links can support prefetching prefixes of directly related videos after a cached video is played.Neighbor prefetching achieves nearly the hit ratio of caching the most popular videos while requiring significantly less communication overhead.
B. Can Peer-to-Peer Save YouTube?
The paper finds that conventional client-server delivery limits scalability, while direct peer-to-peer delivery faces severe churn and removal-control challenges; grouping related videos offers a possible alternative.
- Scalability: YouTube and similar services use client-server architecture, which restricts scalability despite peer-to-peer overlays scaling with larger user bases.The contrast is framed around peers contributing bandwidth to serve other users.
- Peer-to-peer challenges: Short videos, rapid successive viewing, and the huge video population make direct peer-to-peer delivery challenging.These properties imply high overlay churn and very small overlays.
- Related-video grouping: Grouping related videos as portions of one larger video could make the peer-to-peer overlay larger and more stable.Clients could download and upload related portions when bandwidth and storage permit.
- Potential benefit: The proposed grouping behavior can significantly reduce server bandwidth consumption and increase system scalability.The passage presents this as a consequence of clients sharing downloaded portions with interested peers.
- Content control: Peer-to-peer delivery would worsen the challenge of promptly removing videos that violate terms of use.Copyright-protected and illegal content are identified as examples requiring attention before adopting this model.
- Conclusion: The paper concludes that YouTube has unique length, access-pattern, and growth characteristics that create both optimization challenges and opportunities.Its social network of related videos is identified as a basis for novel caching and peer-to-peer strategies.