Source-linked AI summary
Twitter mood predicts the stock market
Johan Bollen, Huina Mao, Xiao-Jun Zeng
TL;DR
Can public mood measured from Twitter predict stock-market changes? The paper compares sentiment and multidimensional mood signals from daily tweets with DJIA values, finding that specific mood dimensions—especially Calm—improve prediction while others do not.
Problem
Reliable, scalable, and timely measures of public mood are needed to assess whether collective sentiment predicts stock-market changes.
Method
The study analyzes large-scale daily Twitter feeds with OpinionFinder and GPOMS, then relates the resulting mood time series to DJIA changes using Granger causality and nonlinear prediction models.
Results
Only selected mood dimensions predict DJIA changes: Calm shows significant Granger causality, while OpinionFinder and other tested GPOMS dimensions do not; adding Calm achieves 86.7% direction accuracy and 1.83% MAPE.
Takeaways & Limitations
Twitter-based public-mood analysis can provide a fast, large-scale way to measure mood dimensions relevant to particular socioeconomic indicators.
Takeaways & Limitations
The analysis primarily reflects predominantly English-speaking Twitter users located in the United States, creating potential geographical and cultural sampling errors.
Abstract
from arXiv · showhide
Behavioral economics tells us that emotions can profoundly affect individual behavior and decision-making. Does this also apply to societies at large, i.e., can societies experience mood states that affect their collective decision making? By extension is the public mood correlated or even predictive of economic indicators? Here we investigate whether measurements of collective mood states derived from large-scale Twitter feeds are correlated to the value of the Dow Jones Industrial Average (DJIA) over time. We analyze the text content of daily Twitter feeds by two mood tracking tools, namely OpinionFinder that measures positive vs. negative mood and Google-Profile of Mood States (GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate the resulting mood time series by comparing their ability to detect the public's response to the presidential election and Thanksgiving day in 2008. A Granger causality analysis and a Self-Organizing Fuzzy Neural Network are then used to investigate the hypothesis that public mood states, as measured by the OpinionFinder and GPOMS mood time series, are predictive of changes in DJIA closing values. Our results indicate that the accuracy of DJIA predictions can be significantly improved by the inclusion of specific public mood dimensions but not others. We find an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA and a reduction of the Mean Average Percentage Error by more than 6%.
I. INTRODUCTION
The introduction frames public mood as a potentially important driver of stock-market values and motivates scalable, timely measurement through large-scale Twitter data. The paper therefore tests whether daily Twitter sentiment and mood dimensions can predict stock-market movements.
- Emotions and mood influence human and financial decision-making, making it reasonable to investigate whether public sentiment drives stock-market values.
- Representative public-mood surveys are expensive and time-consuming, creating a need for reliable, scalable, early assessments suitable for stock-market prediction.
- The paper investigates whether public sentiment in large-scale collections of daily Twitter posts can be used to predict the stock market.
II. RESULTS · A. Data and methods overview
The study analyzes nearly 9.9 million public tweets from 2008 to construct daily public-mood time series. Its three-phase methodology validates OpinionFinder and GPOMS measures, tests their relationship with the DJIA, and predicts DJIA values using a neural network.
- A. Data and methods overview: 9,853,498 public tweets from February 28 to December 19, 2008 were posted by approximately 2.7 million users.Each record included an identifier, GMT+0 submission time, submission type, and tweet text limited to 140 characters.
- A. Data and methods overview: Tweets were cleaned by removing stop-words and punctuation, then grouped by submission date.The records provide the text and timestamps needed to create daily collections.
- A. Data and methods overview: OpinionFinder measured positive versus negative mood, while GPOMS measured six mood dimensions from daily tweet collections.The two tools were applied in the first methodological phase.
- A. Data and methods overview: The two tools produced seven public-mood time series: one from OpinionFinder and six from GPOMS.Each series represented a potentially different aspect of public mood on a given day.
- A. Data and methods overview: The methodology comprised three phases: mood-series creation and validation, Granger causality analysis, and DJIA prediction with a Self-Organizing Fuzzy Neural Network.The phases used election and Thanksgiving data, tested DJIA–mood relationships, and trained predictions from mood combinations.
- A. Data and methods overview: Mood-series creation and validation covered October–December 2008, Granger analysis covered August–December 2008, and DJIA prediction used a Self-Organizing Fuzzy Neural Network.The methodology diagram specifies these time windows and corresponding datasets.
B. Generating public mood time series: OpinionFinder and GPOMS
The study generates public mood time series using OpinionFinder’s positive–negative sentiment measure and GPOMS’s six-dimensional mood framework. The two series are normalized to comparable z-scores using local sliding-window statistics.
- OpinionFinder: OpinionFinder measures sentence-level emotional polarity and derives daily Twitter mood from the ratio of positive to negative tweets.It uses the OpinionFinder lexicon for sentiment analysis of large tweet collections.
- GPOMS: GPOMS measures public mood across 6 dimensions: Calm, Alert, Sure, Vital, Kind, and Happy.It was created to capture mood’s multidimensional structure beyond OpinionFinder’s binary sentiment distinction.
- GPOMS: 964 associated terms expanded the original 72-term POMS questionnaire into a Twitter mood lexicon.The expansion used word co-occurrences in 2.5 billion 4- and 5-grams derived from approximately 1 trillion Web word tokens.
- Time-series normalization: Both mood time series are normalized to z-scores using local means and standard deviations within a sliding window of k days before and after each date.This places the series around zero mean on a scale of 1 standard deviation.
C. Cross-validating OF and GPOMS time series against large socio-cultural events
OpinionFinder and GPOMS captured distinct public-mood responses to the 2008 presidential election and Thanksgiving. OpinionFinder correlated significantly with GPOMS’s Sure, Vital, and Happy dimensions, but not Calm, Alert, or Kind.
- Event-based validation: The validation used tweets from October 5 to December 5, 2008, spanning the U.S. presidential election and Thanksgiving.These events were selected because they could produce unique, significant, and complex effects on public mood.
- Event-based validation: OpinionFinder detected a significant but short-lived uptick in positive sentiment on both the presidential election and Thanksgiving.The election occurred November 4, 2008, and Thanksgiving occurred November 27, 2008.
- Event-based validation: GPOMS showed a differentiated response around the election: Calm dropped before election day, while Vital, Happy, and Kind increased on election day.The Calm reversal indicated reduced public anxiety, while the other increases indicated an energized, happy, and friendly public.
- Cross-method comparison: Multiple regression found OpinionFinder significantly correlated with GPOMS’s Sure, Vital, and Happy dimensions, but not Calm, Alert, or Kind.Thus, some GPOMS dimensions overlap with OpinionFinder, whereas others may capture different components of public mood.
D. Bivariate Granger Causality Analysis of Mood vs. DJIA prices
Bivariate Granger causality analysis tested whether Twitter mood time series predicted daily DJIA changes using lagged linear models. Calm was the only GPOMS dimension with a significant predictive relationship, especially at a three-day lag, although notable deviations occurred.
- Method: The analysis compared a baseline model using only lagged DJIA changes with a second model adding lagged mood and DJIA values.The DJIA series was defined as the day-to-day difference, D_t = DJIA_t − DJIA_t−1.
- Data and period: 342,255 tweets and daily DJIA data from February 28 to November 3, 2008 were analyzed, excluding election and Thanksgiving responses.The DJIA series omitted weekends and holidays, yielding 64 trading days.
- Results: Calm showed the highest Granger-causality relationship with DJIA changes across lags of 2–6 days, with p-values < 0.05.The other four GPOMS dimensions did not show significant causal relationships with DJIA changes.
- Limitations: Calm generally tracked subsequent DJIA movements, but it failed to follow the October 13 surge of more than 3 standard deviations.The Calm curve remained relatively flat during the surge before resuming its tracking of DJIA changes.
- Results: p = 0.009 was observed for Calm at lag n = 3 during August 1 to October 30, 2008.This shorter-period result was significantly lower than the corresponding value for February 28 to November 3, 2008.
E. Non-linear models for emotion-based stock prediction
A Self-organizing Fuzzy Neural Network models nonlinear relations between historical DJIA values and public mood dimensions. Adding Calm produced the strongest test performance, while combining Calm and Happy supported a nonlinear mood–DJIA relationship.
- Model and inputs: The SOFNN is a five-layer hybrid network that self-organizes its neurons while learning nonlinear stock-market time series.The model was selected because related SOFNNs have been used to decode nonlinear stock-market data and predict its values.
- Model and inputs: The model predicts DJIA values from combinations of the previous three days’ DJIA and mood values, with historical inputs scaled to [0,1].The choice n = 3 followed a decline in the Calm–DJIA Granger-causal relation at n = 4.
- Prediction results: Adding OpinionFinder sentiment did not improve prediction accuracy relative to the historical-DJIA baseline I0.This result was consistent with the Granger-causality analysis.
- Prediction results: 1.83% MAPE and 86.7% direction accuracy were achieved by adding Calm through input I1, outperforming the reported alternatives.The comparison values were a maximum MAPE of 2.13%, 1.95% for IOF, 73.3% direction accuracy for IOF, and 46.7% for I1,3.
- Statistical assessment: 87.6% accuracy in predicting DJIA up and down movements had a 0.32% probability of occurring by chance over 15 trials.The calculation assumes a 50% success probability on each trial.
- Nonlinear effects: A p-value of 0.66 and F-statistic of 0.53 showed that the linear Calm–Happy combination performed worse than Calm alone, although the SOFNN improved with both dimensions.The authors interpret this contrast as evidence of a nonlinear relationship among mood dimensions.
III. DISCUSSION
The discussion concludes that large-scale Twitter feeds can track public mood and that some mood dimensions relate to DJIA values, while noting important limitations in the analysis. It also highlights implications for sentiment-tracking tools and subjective well-being surveys.
- Contributions: Large-scale Twitter feeds can track changes in public mood using relatively simple text-processing techniques, and public mood is correlated or predictive of DJIA values.The study finds that mood changes respond to socio-cultural drivers in differentiated ways, although the passage does not specify which dimensions are informative.
- Implications: The findings have implications for sentiment-tracking tools and self-reported subjective well-being surveys.The passage notes that such surveys are relatively expensive and time-consuming and may not measure mood dimensions relevant to assessing particular outcomes.
- Limitations: The analysis is not restricted to a particular geographic location or subset of the world’s population, although Twitter users during the observed period were predominantly English-speaking and located in the US.The authors identify this population coverage as an important factor for future research, while noting that US stock markets may be affected by individuals worldwide.