Source-linked AI summary
Predicting Financial Markets: Comparing Survey, News, Twitter and Search Engine Data
Huina Mao, Scott Counts, Johan Bollen
TL;DR
The paper addresses the difficulty of comparing online sentiment measures when prior studies used isolated data sources and financial targets. It surveys indicators from surveys, news, Google searches, and Twitter, then compares their predictive value across market outcomes. Survey measures are generally weaker after controls, while weekly Google searches and lagged Twitter indicators show predictive value; the paper notes that these relationships and their broader applicability remain incompletely understood.
Problem
Prior studies typically used a single data source and sentiment measure, making it unclear which indicators best capture investor-mood signals for financial prediction.
Method
The paper collects six sentiment indicators from surveys, Twitter, news, and Google search data and evaluates them against DJIA, trading volumes, VIX, and gold prices.
Results
Weekly GIS has predictive value, while lagged TIS and TV-FST are very statistically significant predictors of daily market returns; DSI is not significant after controlling for other mood indicators and VIX.
Takeaways & Limitations
Twitter’s two sentiment indicators showed stronger daily-return predictability than survey sentiment and news-media analysis, while weekly GIS improved forecasting accuracy.
Takeaways & Limitations
The relationships, limitations, and general predictive applicability of different data sources and sentiment measures remain unclear, requiring continued research.
Abstract
from arXiv · showhide
Financial market prediction on the basis of online sentiment tracking has drawn a lot of attention recently. However, most results in this emerging domain rely on a unique, particular combination of data sets and sentiment tracking tools. This makes it difficult to disambiguate measurement and instrument effects from factors that are actually involved in the apparent relation between online sentiment and market values. In this paper, we survey a range of online data sets (Twitter feeds, news headlines, and volumes of Google search queries) and sentiment tracking methods (Twitter Investor Sentiment, Negative News Sentiment and Tweet & Google Search volumes of financial terms), and compare their value for financial prediction of market indices such as the Dow Jones Industrial Average, trading volumes, and market volatility (VIX), as well as gold prices. We also compare the predictive power of traditional investor sentiment survey data, i.e. Investor Intelligence and Daily Sentiment Index, against those of the mentioned set of online sentiment indicators. Our results show that traditional surveys of Investor Intelligence are lagging indicators of the financial markets. However, weekly Google Insight Search volumes on financial search queries do have predictive value. An indicator of Twitter Investor Sentiment and the frequency of occurrence of financial terms on Twitter in the previous 1-2 days are also found to be very statistically significant predictors of daily market log return. Survey sentiment indicators are however found not to be statistically significant predictors of financial market values, once we control for all other mood indicators as well as the VIX.
I. INTRODUCTION
The paper motivates comparing survey, news, search, and Twitter indicators because prior studies used different data sources and financial targets, leaving their relative predictive value unclear.
- Behavioral finance emphasizes social mood and emotional factors as important influences on financial decision-making.
- Surveys measure investor mood but are expensive and vulnerable to responder, individual, social, and groupthink biases.
- Online sentiment analysis may provide faster and more cost-effective measures than large-scale surveys, with evidence of predictive validity in several socioeconomic domains.
- Prior financial-prediction studies examined news, search, and social-media data, including links to returns, trading volumes, volatility, and stock fluctuations.
- The central gap is that it remains unclear which indicators and data sources most effectively capture investor-mood signals for financial prediction.
- The paper collects multiple sentiment indicators and evaluates their predictive value for DJIA prices, trading volumes, VIX, and gold prices.
A. Survey Data
The paper constructs sentiment measures from surveys, news headlines, and Google search activity, using established survey sources and financial-language processing procedures.
- Survey Data: Investor Intelligence provides weekly bullish, bearish, or correction classifications from more than one hundred market newsletters, while DSI supplies daily sentiment readings across active US markets.
- News Media: The news dataset covers eight major financial media outlets followed through their Twitter accounts to capture recent and featured news.
- News Media: Negative News Sentiment divides negative financial terms by headline length and averages the resulting emotional ratios across each day’s news articles.
- News Media: The news measure uses the Loughran–McDonald financial negative lexicon, which the paper describes as better suited to financial text than the Harvard IV dictionary.
- Search Engine Data: Google search activity is measured weekly for 26 financial terms expanded from seed queries through Google Insights for Search.
D. Social Media Data
The paper uses sampled public tweets to construct Twitter mood indicators and compares tweet volumes with Google search volumes for the same financial terms.
- Social Media Data: The study uses a 15%-30% random sample of public tweets posted daily from July 2010 to September 2011.
- Social Media Data: Two Twitter indicators are defined: Twitter Investor Sentiment and Tweet Volumes of Financial Search Terms.
- 1) Twitter Investor Sentiment:: Twitter Investor Sentiment is based on counts of bullish and bearish tweets for each day.
- 2) Tweet Volumes of Financial Search Terms (TV-FST):: TV-FST and GIS are aligned by averaging daily tweet volumes weekly, then averaging the separate weekly series across the 26 financial terms.
- 2) Tweet Volumes of Financial Search Terms (TV-FST):: The weekly TV-FST and GIS series have a statistically significant Pearson correlation of 0.62 (p < 0.01).
- 2) Tweet Volumes of Financial Search Terms (TV-FST):: TV-FST began increasing five weeks before GIS during a period preceding the DJIA’s 1864-point decline, suggesting earlier detection of negative sentiment by Twitter.
E. Economic and Financial Market Data
The paper compares Google Insight Search volumes with financial indicators using correlations, trends, and cross-correlations over market data spanning varied conditions. Search volumes correlate positively with VIX and trading volume, negatively with DJIA prices, and can lead some financial series.
- Data and analysis: The study computes pair-wise correlations after transforming all time series to log scale, then examines top-correlated terms and cross-correlation lags.The cross-correlation analysis also includes Investor Intelligence for comparison with search volumes.
- Correlation results: GIS search volumes generally co-occur with VIX and trading-volume peaks and sometimes precede peaks in DJIA and gold series.The figure compares GIS trends and log-scale scatter plots against VIX, DJIA closing values, gold price, and DJIA trading volume.
- Correlation results: Search volumes show high positive correlations with VIX (γ = 0.88) and trading volume (γ = 0.70), but a high negative correlation with DJIA price (γ = −0.77).The correlation between gold price and “gold” search volume is γ = 0.45.
- Interpretation: Financial-term search volumes reflect VIX fluctuations, suggesting they may serve as a computational gauge of investor fear.VIX is described as a widely used measure of market risk and the “investor fear gauge.”
- Lagged relationships: GIS leads DJIA most strongly at positive lags, also leads VIX especially at k = [+1, +3] weeks, while gold searches do not consistently lead gold prices.The authors speculate that gold-search behavior may reflect a nonlinear interaction with absolute gold price levels.
B. Granger Causality Analysis
The paper uses Granger causality tests to evaluate whether lagged search-volume and survey series improve forecasts of financial indicators. Search volumes help predict several indicators, whereas Investor Intelligence predicts VIX only in the reverse direction.
- Testing procedure: The Granger causality test evaluates whether adding one time series improves forecasts of another by testing whether the null of no predictive usefulness can be rejected.An F-test is used to examine rejection of the null hypothesis.
- Interpretive limitation: Granger causality indicates statistically significant lagged correlation, not actual causation.The paper cautions that predictive precedence does not establish that the earlier series causes the later one.
- Testing procedure: Table IV tests both directions across positive and negative lags, representing whether each series may Granger-cause the other.The table reports statistical significance as p-values over lags of 1, 2, and 3 weeks.
- Search-volume results: Adding GIS helps predict VIX and DJIA, while GIS from the previous 1 to 2 weeks significantly Granger-causes trading volume.GIS also has very significant predictive relationships with gold at lags of 2 and 3 weeks.
- Survey comparison: The relationship between Investor Intelligence and VIX runs only from VIX to Investor Intelligence; adding survey data does not help predict VIX.The reported direction is VIX→II.
C. Forecasting Analysis
The analysis tests whether adding sentiment indicators improves one-step-ahead forecasts of DJIA, trading volume, and VIX relative to historical-value baselines. Weekly search volumes generally reduce prediction error, improve direction accuracy for DJIA and VIX, and perform unevenly during volatile periods.
- MAPE prediction error decreases for VIX, DJIA, and trading-volume forecasts when weekly search volumes are added.
- Direction accuracy improves for DJIA and VIX forecasting, but not for trading-volume forecasting, after adding search volumes.
- Baseline forecasts outperform the advanced model during several weeks, underscoring the difficulty of prediction even with statistically significant Granger-causal data.
A. Correlation Analysis
The correlation analysis compares daily survey, Twitter, news, and tweet-volume indicators with DJIA movements and related market measures. Non-survey indicators, especially TIS and TV-FST, rise before several periods of falling DJIA prices, while survey relationships can lag or differ in direction.
- Daily analysis covers TIS, TV-FST, NNS, and DSI, using Twitter and survey data unavailable at the weekly search-volume frequency.
- TV-FST correlates negatively with DSI and TIS but positively with NNS, suggesting that it functions as a bearish or negative sentiment indicator.
- TIS is positively correlated with market log returns and negatively correlated with VIX, while DSI correlates positively with DJIA closing values and log return.
- The plotted series invert TIS and DSI so that upward movement consistently represents negative sentiment alongside NNS and TV-FST.
- TIS and TV-FST rise before several DJIA declines, and non-survey indicators generally precede falling prices despite considerable daily noise.
B. Granger Causality Analysis
The Granger-causality analysis tests whether daily sentiment indicators predict DJIA log returns. Twitter indicators and negative news sentiment show predictive relationships, whereas the Daily Sentiment Index does not.
- TV-FST has an insignificant correlation with daily log returns, but the selected TV-FST′ has correlation -0.30 with p-value < 0.01.
- Bidirectional Granger causation is statistically significant between log returns and TIS, NNS, and TV-FST′, subject to specified lag exceptions.
- No statistically significant Granger causation is observed between DSI and log returns.
- The results identify Twitter- and news-derived indicators as predictive of DJIA log returns, unlike the survey-based DSI.
C. Multiple Regression Analysis
Multiple regression evaluates daily log returns using sentiment indicators, lagged returns, and VIX. Adding sentiment indicators raises adjusted R2, with Twitter measures remaining strong predictors after controls, while DSI is not statistically significant.
- The regression uses four sentiment indicators, past log returns, and VIX, with n = 7 days and standardized inputs.
- Adjusted R2 improves from 0.092 to 0.200 after sentiment indicators are added, accounting for an additional 11% of log-return variation.
- DSI is not statistically significant after controls, whereas TIS and TV-FST′ are very significant predictors at lags of 1 to 2 days.
- The 30-day forecasting test finds improved direction accuracy and MAPE for DJIA, VIX, and volume, except for volume MAPE, but improvements are not highly significant.
- The authors attribute weak forecasting improvements partly to extreme volatility and the limited suitability of simple linear models for complex market interactions.
V. CONCLUSION
The paper compares sentiment indicators from surveys, Twitter, news, and Google searches across multiple financial indexes and timescales. It finds that Google search volumes improve weekly forecasting, while Twitter indicators provide strong daily predictive signals after controlling for other mood indicators and the VIX.
- Motivation: The paper frames this comparison as necessary because prior studies generally used one data source and one financial index, leaving relationships among mood indicators unclear.
- Scope: The study evaluates six sentiment indicators from surveys, Twitter, news media, and Google search data against financial indexes including DJIA, trading volumes, VIX, and gold.The indicators are DSI, II, TIS, TV-FST, NNS, and GIS.
- Weekly analysis: Weekly Google search volumes for financial terms significantly correlate with DJIA closing indicators, whereas Investor Intelligence surveys do not.
- Weekly analysis: Adding weekly GIS financial-search volumes improves forecasting accuracy, especially when the DJIA declines and the VIX signals high volatility.The paper highlights August 2011 as an example.
- Daily analysis: After controlling for other mood indicators and the VIX, previous 1–2 days of TIS and TV-FST are very statistically significant predictors of daily market returns.NNS is also significant but less predictive than the two Twitter indicators.
- Daily analysis: The study reports that Twitter financial-term volumes increased weeks before Google volumes ahead of the DJIA’s sharp decline, indicating a potential Twitter efficiency gain over GIS.