Source-linked AI summary
Predicting the Future with Social Media
Sitaram Asur, Bernardo A. Huberman
TL;DR
The paper addresses whether largely untapped social-media content can support predictions of real-world outcomes. It uses Twitter chatter about movies to forecast box-office revenues, then tests whether sentiment adds predictive power. Tweet-rate models outperform the Hollywood Stock Exchange, while sentiment improves predictions only after release.
Problem
The paper asks whether social-media chatter can be harnessed to make specific predictions about real-world outcomes without market mechanisms.
Method
The authors use Twitter movie chatter, tweet rates, linear regression, and sentiment classification to forecast box-office revenues.
Results
Tweet-rate predictions outperform the Hollywood Stock Exchange, and sentiment improves predictions after a movie has released.
Takeaways & Limitations
Social-media attention can provide a powerful indicator of future outcomes, with sentiment adding value after release.
Abstract
from arXiv · showhide
In recent years, social media has become ubiquitous and important for social networking and content sharing. And yet, the content that is generated from these websites remains largely untapped. In this paper, we demonstrate how social media content can be used to predict real-world outcomes. In particular, we use the chatter from Twitter.com to forecast box-office revenues for movies. We show that a simple model built from the rate at which tweets are created about particular topics can outperform market-based predictors. We further demonstrate how sentiments extracted from Twitter can be further utilized to improve the forecasting power of social media.
I. INTRODUCTION
The paper investigates whether Twitter chatter can predict movie box-office performance and outperform established market-based predictors. It also examines whether tweet sentiment adds predictive value before or after release.
- Motivation: The study asks whether social-media content can serve as an indicator of real-world outcomes without instituting market mechanisms.The authors frame social media as a source of collective information that can support quantitative predictions.
- Research focus: The paper predicts movie box-office revenues using chatter from Twitter, a rapidly growing micro-blogging network.Movies were selected because users discuss them extensively and their outcomes are directly observable through box-office revenue.
- Research focus: The model uses the rate at which movie-related tweets are generated to forecast box-office revenue.The paper focuses on attention, popularity, viral marketing, and pre-release hype as predictors of performance.
- Results: Tweet-rate predictions consistently outperform those produced by the Hollywood Stock Exchange.The Hollywood Stock Exchange is described as an industry-standard information market for movie forecasts.
- Results: Sentiment improves tweet-rate-based box-office predictions only after movies are released.The analysis distinguishes positively and negatively oriented tweets using text classifiers.
II. RELATED WORK
Prior work studied Twitter’s social structure, user intentions, and word-of-mouth advertising, but did not analyze Twitter’s predictive use. Related forecasting research used blogs or movie metadata rather than Twitter content.
- Twitter research: Earlier Twitter studies examined sparse interaction networks and community structure, including different user intentions.These studies focused on describing Twitter behavior rather than forecasting outcomes.
- Twitter research: Research on Twitter-based word-of-mouth advertising analyzed postings, brands, products, and sentiment changes without performing predictive analysis.The paper identifies this predictive focus as the distinction from prior Twitter work.
- Forecasting research: Blog-mining research had used online mentions to predict spikes in book sales.This provides a related example of social content being connected to commercial outcomes.
- Forecasting research: Most prior movie-sales forecasting used movie metadata such as genre, rating, running time, release date, screens, and cast.The paper instead studies Twitter chatter as a forecasting signal.
- Twitter background: At the time of the study, Twitter had a large user base, with users posting short updates and propagating content through followers and retweets.These platform properties provide the underlying environment for analyzing movie-related chatter.
IV. DATASET CHARACTERISTICS
The dataset tracks movie-related Twitter activity around theatrical release and examines how attention evolves across movies and users. The collection contains 2.89 million tweets from 24 movies, with chatter generally peaking near release and then fading.
- Dataset construction: 2.89 million tweets about 24 movies were collected over three months using hourly Twitter feed data and movie-title keywords.The crawl retained timestamps, authors, and tweet text for analysis.
- Dataset construction: The critical period runs from one week before release through two weeks after release.This window captures promotional activity before release and the spread of initial opinions afterward.
- Dataset construction: The study includes Friday releases in wide release, while excluding titles whose tweets could not be reliably identified.For initially limited-release movies, collection began when they entered wide release.
- Dataset scale: The 24-movie critical-period dataset contains 2.89 million tweets from 1.2 million users.This passage reports the total number of tweets and unique users represented in the analyzed data.
- Attention patterns: Movie tweet activity is busiest around release and then invariably fades, paralleling the concentration of box-office revenue around opening weekend.The observed time-series pattern links attention timing with the period of strongest theatrical revenue.
- Attention patterns: Tweets per unique author remain fairly consistent between 1 and 1.5 across the critical period.The author distribution also indicates that most authors discuss only a few movies.
V. ATTENTION AND POPULARITY
The section examines how Twitter attention and promotional activity develop around movie releases and whether they predict box-office performance. URL activity is higher before release, but URLs and retweets show limited predictive power for relative performance.
- Section focus: The analysis studies how attention and popularity are generated for movies and how they affect real-world performance.It focuses on publicity, word-of-mouth cascades, and pre-release hype on Twitter.
- Pre-release attention: Movies receive more URL-containing tweets in the week before release than afterward.Retweet percentages remain similar across the three weeks examined and represent a significant minority of movie tweets.
- Attention and performance: URLs and retweets correlate moderately positively with box-office performance but have low adjusted R2 values.The results indicate that these publicity features are not very predictive of movies’ relative performance.
B. Prediction of first weekend Box-office revenues
This section tests whether pre-release Twitter activity can predict opening-weekend box-office revenue. A regression model using tweet-rate information produces accurate predictions and outperforms the Hollywood Stock Exchange and an earlier news-based model.
- Research question: The study asks whether pre-release movie tweets can accurately predict opening-weekend box-office revenue.The prediction uses tweets referring to movies before their release.
- Tweet-rate prediction: 0.90 was the correlation coefficient between average pre-release tweet-rate and box-office gross.A least-squares linear regression over the week before release achieved adjusted R2 = 0.80 with p-value 3.65e-09.
- Prediction model: The regression uses seven daily pre-release tweet-rate variables plus the number of release theaters.All predictors were available before release, while the target was opening-weekend box-office revenue.
- Comparison with HSX: The tweet-rate timeseries model outperforms the HSX-based model for predicting movie box-office performance.Predicted and actual values are compared in Fig. 6.
- Comparison with news-based prediction: The tweet-based predictors outperform the earlier combined IMDB-and-news model, whose reported R2 was 0.788.The comparison uses AMAPE and score values, but the tweet dataset covers fewer movies than the earlier study.
C. Predicting HSX prices
The section evaluates whether Twitter data can forecast the HSX price of a movie stock at the end of its opening weekend. It uses historical HSX prices and tweet-rates as predictors.
- Prediction target: The experiment predicts each movie’s HSX stock price at the end of its opening weekend.HSX adjusts movie-stock prices after the first weekend to reflect actual box-office gross.
- Predictors: The models use historical HSX prices and tweet-rates individually for the week before release.The passage describes an experiment on the movies considered in the study.
D. Predicting revenues for all movies for a given weekend
This section extends forecasting from opening-weekend revenue to revenue for all movies over a particular weekend. It uses seven-day tweet-rate timeseries because tweet discussions continue after release, unlike HSX movie-stock histories.
- Weekend revenue forecasting: The study predicts box-office revenue for all movies over a particular weekend rather than only opening-weekend revenue.The results cover three weekends in January and one weekend in February.
- Results: The section includes a table for predicting second-weekend box-office gross.The table is labeled as Table VIII.
- Results: The analysis reports coefficient-of-determination values for tweet-rate timeseries across different weekends.The table covers weekend-specific prediction results.
- Prediction window: The model uses tweet-rate over the seven days before each weekend.Tweets remain available after release, whereas HSX delists movie stocks after four weeks.
VI. SENTIMENT ANALYSIS
The paper uses sentiment analysis to distinguish positive, negative, and neutral movie tweets, then examines how sentiment contributes to revenue forecasting. Sentiments improve tweet-rate predictions after release, when post-release opinions become more informative.
- VI. SENTIMENT ANALYSIS: Sentiment analysis classifies movie tweets as Positive, Negative, or Neutral using a LingPipe DynamicLMClassifier.The classifier uses an n-gram language model with n=8 after tweet preprocessing.
- VI. SENTIMENT ANALYSIS: The classifier achieved 98% cross-validation accuracy on the training set before labeling tweets for all movies.
- VI. SENTIMENT ANALYSIS: The analysis evaluates subjectivity and polarity as sentiment measures across the movies’ critical periods.Figures 7 and 8 report movie subjectivity and polarity values.
A. Subjectivity
The subjectivity and polarity measures capture how movie opinions change around release. Post-release sentiment generally becomes more pronounced, and adding polarity to tweet-rate models improves second-week revenue prediction.
- A. Subjectivity: Subjectivity generally increases after release, with more sentiments appearing in the two post-release weeks than during the pre-release week.The measure is based on the ratio of subjective positive and negative tweets to objective tweets.
- B. Polarity: Polarity measures the ratio of positive to negative tweets, with higher values indicating relatively more positive sentiment.
- B. Polarity: The Blind Side’s polarity rose from 5.02 to 9.65, while its box-office revenue increased from 34M to 40.1M in the following week.
- B. Polarity: New Moon’s polarity fell from 6.29 to 5, alongside a box-office decline from 142M to 42M in the following week.
- B. Polarity: Adding sentiment to tweet-rate regression improved second-week prediction to 0.92 with average tweet-rate and 0.94 with tweet-rate timeseries.The tweet-rate remained more important than sentiment, and the regression coefficients were highly significant.
VII. CONCLUSION
The paper shows that Twitter chatter can forecast movie box-office revenue and that sentiment analysis further improves predictions after release. It also argues that the approach can extend beyond movie revenues to other consumer-interest outcomes.
- VII. CONCLUSION: Using nearly 3 million tweets, the authors build a linear regression model that predicts movie box-office revenues before release.
- VII. CONCLUSION: The Twitter-based predictions outperform those of the Hollywood Stock Exchange.
- VII. CONCLUSION: Sentiment analysis improves revenue predictions after a movie has been released.
- VII. CONCLUSION: The method can be extended from movie revenue prediction to other consumer-interest products and outcomes.The paper names product ratings, agenda setting, and election outcomes as possible extensions.
VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA
The appendix generalizes the forecasting approach to products discussed over time in reviews, comments, and blogs. It models revenue from attention, sentiment polarity, and distribution-related variables using linear regression.
- VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA: The generalized model begins with product data collected over time from reviews, user comments, and blogs.Collecting data over time allows the rate of chatter to be measured.
- VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA: The model uses least-squares linear regression with attention, sentiment polarity, and distribution as parameters.
- VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA: Revenue is expressed as y = βa *A + βp *P + βd *D + ǫ, where the β values are regression coefficients.
- VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA: Attention captures social-media buzz, while polarity captures opinions and views disseminated through social media.
- VIII. APPENDIX: GENERAL PREDICTION MODEL FOR SOCIAL MEDIA: The attention coefficient was most significant in the experiments, while polarity became more important after movie release and improved prediction accuracy.For movies, distribution is represented by the number of theaters; for other products, it can represent market availability.