Source-linked AI summary
Stock Market Forecasting Based on Text Mining Technology: A Support Vector Machine Method
Yancong Xie, Hongxun Jiang
TL;DR
The paper asks whether Chinese online financial news can help predict stock movements in a market where investors may be susceptible to external information. It combines news text and stock data in support vector models, reporting strong price-prediction fitting but weaker tendency classification. It also proposes a way to evaluate online news-source influence.
Problem
The paper examines whether online news can help predict Chinese stock movements, a phenomenon the authors say has received limited research attention.
Source: §1 · p. 2Method
The authors combine text mining of online news with stock data and use support vector methods to predict stock tendency and price.
Source: §1 · p. 2Results
The authors report outstanding stock-price prediction but insufficient stock-tendency prediction, and find that parameter g is the main factor in parameter search.
Source: §5 · p. 10Takeaways & Limitations
The proposed source-weight approach evaluates online sources by their real effect on investors’ trading decisions.
Source: §5 · p. 10When news volume is low, the SVM prediction is not good enough, and even simple linear regression can do better.
Source: §5 · p. 10
Abstract
from arXiv · showhide
News items have a significant impact on stock markets but the ways are obscure. Many previous works have aimed at finding accurate stock market forecasting models. In this paper, we use text mining and sentiment analysis on Chinese online financial news, to predict Chinese stock tendency and stock prices based on support vector machine (SVM). Firstly, we collect 2,302,692 news items, which date from 1/1/2008 to 1/1/2015. Secondly, based on this dataset, a specific domain stop-word dictionary and a precise sentiment dictionary are formed. Thirdly, we propose a forecasting model using SVM. On the algorithm of SVM implementation, we also propose two-parameter optimization algorithms to search for the best initial parameter setting. The result shows that parameter G has the main effect, while parameter C's effect is not obvious. Furthermore, support vector regression (SVR) models for different Chinese stocks are similar whereas in support vector classification (SVC) models best parameters are quite differential. Series of contrast experiments show that: a) News has significant influence on stock market; b) Expansion input vector for additional situations when that day has no news data is better than normal input in SVR, yet is worse in SVC; c) SVR shows a fantastic degree of fitting in predicting stock fluctuation while such result has some time lag; d) News effect time lag for stock market is less than two days; e) In SVC, historic stock data has a most efficient time lag which is about 10 days, whereas in SVR this effect is not obvious. Besides, based on the special structure of the input vector, we also design a method to calculate the financial source impact factor. Result suggests that the news quality and audience number both have a significant effect on the source impact factor. Besides, for Chinese investors, traditional media has more influence than digital media.
1. Introduction
Forecasting stocks is difficult because prices reflect many influences and are described by the efficient market hypothesis as following a random-walk pattern. The paper focuses on whether Chinese online financial news can help predict Chinese stock movements.
Forecasting challenge
- Stock forecasting is difficult because prices are influenced by numerous macroeconomic and market factors, while the efficient market hypothesis describes them as following a random walk.The listed influences include political events, firm policies, economic conditions, investor expectations, and other stock markets.Source: §1 · pp. 1–2
Prior approaches
- Research has moved from methods such as moving averages and ARIMA toward nonlinear machine-learning approaches and data beyond historical stock prices.The paper describes text as one such additional data source.Source: §1 · p. 2
Text-based approaches
- Prior text-based approaches either classify positive and negative language or examine external media reports that investors may react to differently.The paper distinguishes customer or market feedback from passive information in mass media.Source: §1 · p. 2
Motivation
- The authors argue that Chinese investors’ short-term speculative trading and susceptibility to external information make news effects on the market worth studying.They note that relatively few studies have addressed this phenomenon and propose using text mining and support vector algorithms to predict Chinese stock movements.Source: §1 · p. 2
2. Research Method
The method combines online news and stock data in a forecasting pipeline of data collection, preprocessing, and machine learning. It constructs text dictionaries and structured inputs, then applies support vector classification and regression to stock tendency and price tasks.
2. Research Method
- The research method has three stages: data collection, data preprocessing, and machine-learning forecasting.The method merges text and market data into input vectors.Source: §2 · p. 2, §2.1 · p. 2
2.1.1. News source selection
- News collection covers 20 Chinese online financial sources across eight news categories, selected with source importance and field coverage in mind.The corpus contains 2,302,692 Chinese news documents dated January 1, 2008, to January 1, 2015.Source: §2.1.1 · p. 3
2.1.2. Stock selection
- The study tests 20 Chinese stocks, prioritizing trading volume while also considering sector, publication date, and stock price.The authors describe high-volume stocks as more mobile and relevant to investors.Source: §2.1.2 · p. 3
2.2. Data Pre-processing
- Each daily input vector combines news-source nodes with stock-data nodes spanning a selected time lag.X denotes the number of news sources and Y the stock-data time lag; the study sets X to 20.Source: §2.2 · p. 3
2.2.1. Text mining
- Text mining builds stop-word and domain-specific sentiment dictionaries, then uses them to form structured inputs for each firm.The process selects 500 significant words and has three experts score words from -5 to +5, continuing until 90% of selected words are scored.Source: §2.2.1 · pp. 3–4
2.2.2. Stock price
- The study uses adjusted close price as its stock-price input rather than close price, which the authors say fluctuates with multiple factors.The paper describes adjusted close as a divided-weight price.Source: §2.2.2 · p. 4
2.2.3. Support vector machine
- The method uses C-SVC with a polynomial kernel for stock tendency and epsilon-SVR with a sigmoid kernel for stock prices.The authors scale inputs because news and market variables have different magnitudes; after scaling, inputs are described as belonging to a standard normal distribution.Source: §2.2.3 · p. 4
3. Experiments and Results
The experiments compare news and stock-history inputs, prediction lags, and parameter-search methods, using Bank of China first and then testing all 20 stocks. Results distinguish SVR’s strong price fitting from weaker SVC tendency accuracy and report differences in input expansion and cross-stock parameter behavior.
3.1.1. Time lag
- News effects lasted fewer than two days, while the authors selected a 10-day lag as the standard for later tendency-prediction experiments.The experiment varied the lag from 1 to 20 days and compared news-plus-stock inputs with stock-only inputs.Source: §3.1.1 · p. 5, Figure 4 · p. 5, Figure 5 · p. 5
3.1.2. Input expansion
- Expansion input generally performed better than original input in SVR, except for stock 601390, while SVC results were entirely opposite.Expansion sets news input nodes to zero on days without news; the authors suggest partial sentiment scoring can make zero a disturbance, especially for classification.Source: §3.1.2 · pp. 5–6, Figure 6 · p. 6, Figure 7 · p. 6
3.1.3. General SVM result
- 98.5429% SCC and 0.00328 MSE were reported for SVR price fitting on Bank of China stock 601988.The authors also report that SVR forecasts sharp changes but may lag by one or two days.Source: §3.1.3 · p. 6, Figure 8 · p. 7
- 59.1734% accuracy was reported for SVC tendency prediction on Bank of China stock 601988.The authors characterize this result as not good.Source: §3.1.3 · p. 7
3.1.4. Parameter optimization
- The parameter search varied c from 1 to 25 and g from 0.01 to 0.30, evaluating each pair by averaged SCC and MSE across 10 random train/test groups.The authors say these averages serve evaluation only and have no statistical meaning.Source: §3.1.4 · p. 7, §3.1.5 · p. 7
3.1.5. Traverse algorithm
- Parameter g was the main model factor, and best parameters differed across stocks for SVC while SVR parameters were similar.The authors report that g near 0 performed much better; for SVC, best g fluctuated across stocks.Source: §3.1.5 · pp. 8–9, Figure 9 · p. 8, Figure 10 · p. 8
3.1.6. Approximate algorithm
- The approximate search used 50 random train/test groups and produced results similar to traversal; traversal was preferred for time complexity.Each group yielded a local optimum, and the authors report that smaller step lengths increase convergence precision.Source: §3.1.6 · p. 9, Figure 11 · p. 9
4. Source Impact Factors
The paper estimates news-source impact by measuring how perturbed source inputs change stock-price predictions, then aggregates those effects across stocks. It reports that news quality and audience size both matter, with newspapers and magazines among the highest-weight sources.
4. Source Impact Factors
- Each source’s weight for a stock is its prediction-price change after adding 100 to that source’s input, relative to the normal prediction.The method compares the perturbed prediction with the normal prediction.Source: §4 · p. 9
- The paper combines stock-level source weights across 20 stocks using trading volume, then scales the resulting market-wide weights.The resulting source weights are presented in Table 5.Source: §4 · pp. 9–10, Table 5 · p. 10
- News quality and audience size both significantly affect source weights, and newspapers and magazines account for most high-weight sources.The authors also identify large audiences as a reason finance portals rank highly, even when they are not the original source of the news.Source: §4 · p. 10
5. Conclusions and Discussions
The authors conclude that text-mining-based SVM forecasting performs especially well for stock prices, while SVC tendency prediction is weaker. They also highlight gamma’s role in parameter search and note that low news volume limits performance.
5. Conclusions and Discussions
- The SVM method performs especially well for specific stock-price prediction, while SVC tendency prediction is not good enough despite showing news-document importance.Source: §5 · p. 10
- Parameter g is the main search factor, and searching it first can increase parameter-search speed.The authors also report that their traverse algorithm is better than the other parameter-finding approach they discuss.Source: §5 · p. 10
- The paper proposes a method to estimate online sources’ influence on investors’ trading decisions using their real effect.The authors judge this approach better than methods based only on audience numbers or specialists’ proposals.Source: §5 · p. 10
- When news volume is low, SVM predictions are not good enough, and even simple linear regression can do better.The authors call for more text sources, improved sentiment evaluation, sufficient text and trading volume, and a larger dataset.Source: §5 · p. 10