Source-linked AI summary
The Pulse of News in Social Media: Forecasting Popularity
Roja Bandari, Sitaram Asur, Bernardo A. Huberman
TL;DR
The paper asks whether news popularity on Twitter can be predicted before publication despite complex social and contextual influences. It constructs article-content features and evaluates regression and classification methods. Classifiers predict popularity ranges with 84% overall accuracy, while source popularity patterns differ between conventional news agencies and leading social-web propagators.
Problem
The paper addresses the difficult gap of predicting social popularity before publication rather than using early popularity measurements.
Method
The study scores source, category, language subjectivity, and named-entity features, then applies regression and classification to predict Twitter spread.
Results
84% overall accuracy was achieved for predicting popularity ranges on Twitter using classifiers and content features.
Takeaways & Limitations
Article content can provide effective popularity ranges even when it cannot predict exact tweet counts, and top Twitter sources are not necessarily conventionally popular agencies.
Takeaways & Limitations
Overlapping Feedzilla categories weakened category-based regression prediction, motivating independently extracted categories with less overlap.
Abstract
from arXiv · showhide
News articles are extremely time sensitive by nature. There is also intense competition among news items to propagate as widely as possible. Hence, the task of predicting the popularity of news items on the social web is both interesting and challenging. Prior research has dealt with predicting eventual online popularity based on early popularity. It is most desirable, however, to predict the popularity of items prior to their release, fostering the possibility of appropriate decision making to modify an article and the manner of its publication. In this paper, we construct a multi-dimensional feature space derived from properties of an article and evaluate the efficacy of these features to serve as predictors of online popularity. We examine both regression and classification algorithms and demonstrate that despite randomness in human behavior, it is possible to predict ranges of popularity on twitter with an overall 84% accuracy. Our study also serves to illustrate the differences between traditionally prominent sources and those immensely popular on the social web.
1 Introduction
News popularity is difficult to predict because articles are time-sensitive and spread amid competition, network complexity, and varying real-world conditions. This study predicts Twitter popularity before publication using article-content features and reports 84% accuracy for popularity ranges.
- News articles must reach many readers quickly because they concern developing events with short lifespans.
- Popularity prediction is difficult because external conditions, social-network structure, member influence, and interactions across web communities all matter.
- Unlike prior studies using early popularity measurements, this work predicts social popularity from article features available before publication.
- The study measures Twitter spread as the number of times a news URL is posted and shared, using articles collected from Feedzilla.
- The features include source, news category, language subjectivity, and named entities mentioned in the article.
- 84% overall accuracy was achieved for popularity ranges using regression and classification methods based only on content features.
2 Related Work
Prior research models information diffusion, social influence, and the temporal spread of online news. Much existing popularity prediction uses early engagement signals, while related work also examines content categories and social marketing messages.
- Information-diffusion research includes stochastic and epidemic models, viral marketing, web attention, and cascading information behavior.
- Controlled music experiments found that social influence shaped popularity outcomes more than song quality among highly rated songs.
- Studies of online news examine temporal spread on Digg and Twitter, while other work studies Facebook news feeds.
- Many popularity-prediction studies rely on early votes, likes, click-throughs, comments, or related engagement measurements.
- Related work on Twitter hashtags studies how content categories vary in spread, with other research examining social marketing messages.
3 Data and Features
The paper builds a dataset and content-based feature space to predict Twitter popularity before publication. It scores article category, subjectivity, named entities, and source history, then evaluates prediction methods against Twitter propagation patterns and source popularity.
- Dataset Description: The dataset combines Feedzilla news feeds with Twitter counts for article links, using a 50-day Twitter history for some feature scores.Articles were collected during August 8–16, 2011, and links were allowed four days to propagate before counting tweets.
- Feature Space: The prediction task uses article content and source information to estimate the number or range of tweets before publication.The study evaluates both regression and classification methods using content-derived feature scores.
- Category Score: Category popularity is represented by a t-density score computed as the average tweets per article link within each category.The category averages are calculated from the first 22,000 dataset points and serve as prior popularity estimates.
- Subjectivity: Subjectivity is encoded as a binary feature distinguishing emotional, personal, subjective language from factual and objective language.Subjective and objective training corpora achieved 99% training accuracy, with manual inspection supporting the final classifications.
- Named Entities and Sources: Named entities are extracted from article titles and summaries, while source scores summarize each publisher’s historical Twitter success.The paper defines named entities as known places, people, or organizations and examines temporal variation in source t-density.
- Temporal Source Patterns: Across sources, tweets and links show weekly cycles, but t-density does not, and individual source t-density can vary substantially over time.Mashable has a more consistent t-density than Blog Maverick; their reported t-density scores are 74 and 178, respectively.
- Source Popularity: NewsKnife ratings correlate positively with the number of links published but have no correlation (-0.05) with tweets per link, whereas the paper’s source scoring correlation is about 0.7.The comparison distinguishes publishing volume from the Twitter propagation rate of individual links.
- Source Popularity: Traditional prominence and Twitter propagation differ sharply: NewsKnife favors Reuters and The Wall Street Journal, while high-t-density sources include Mashable and specialized blogs.The Christian Science Monitor averaged 16 tweets, whereas Mashable averaged nearly 1000, with Mashable’s least popular article receiving 360 tweets.
4 Prediction
The paper evaluates regression and classification using article-derived feature scores to predict Twitter popularity, including tweet counts, popularity classes, and whether articles receive any tweets. Classification reaches 84% overall accuracy, while publication source is the strongest predictor and category overlap limits regression.
- Regression: Regression uses category, subjectivity, named-entity, and source scores as inputs, with linear regression, KNN regression, and SVM regression evaluated.Tweet counts and selected scores are logarithmically transformed before regression because tweet counts are long-tailed.
- Regression: Category-specific regression for Technology reaches R2 = 0.43, compared with an overall regression result of R2 = 0.34 for the improved model.The authors attribute weak category-score prediction to overlap among Feedzilla categories, which are not necessarily disjoint.
- Classification: 84% overall accuracy is achieved when classifying 10,000 articles into low-, medium-, or high-tweet popularity classes.The classes are A (1 to 20 tweets), B (20 to 100 tweets), and C (more than 100 tweets), evaluated with 10-fold cross-validation.
- Classification: Publication source contributes more to popularity prediction than subjectivity, categories, or named entities.A leave-one-feature-out analysis finds that the latter features provide little improvement in predicting Twitter popularity.
- Classification: SVM classification predicts whether an article receives zero or nonzero tweets with 66% accuracy, with source followed by category as the most significant features.Named entities and subjectivity add no information for this prediction, and the classification scheme excludes a zero-tweet class from its popularity classes.
5 Discussion and Conclusion
The study uses article-content features to predict Twitter popularity ranges before publication, achieving strong classification accuracy while revealing source-related differences in social sharing.
- Features: Four content features covered the article source, category, linguistic subjectivity, and named entities.Together, these features represent distinct information available from article content.
- Results: 84% overall accuracy was achieved when classifiers predicted popularity ranges on Twitter.The features were not sufficient to predict an article’s exact number of tweets.
- Predictors: Article source was one of the most important popularity predictors, whereas the Feedzilla category feature performed poorly.The authors attribute the category result partly to overlapping categories supplied by Feedzilla.
- Social sharing: Technology blogs such as Mashable and the Google Blog were widely shared despite not being conventionally prominent news agencies.The comparison is based on retweet counts among Twitter news sources.