Source-linked AI summary
Improving Stock Market Prediction via Heterogeneous Information Fusion
Xi Zhang, Yunjia Zhang, Senzhang Wang, Yuntao Yao, Binxing Fang, Philip S. Yu
TL;DR
Prior stock-prediction methods often use limited information sources or ignore interactions among heterogeneous data and correlated stocks. The paper combines Web events, social-media sentiments, quantitative features, and stock correlations using coupled matrix and tensor factorization, achieving reported effectiveness on China A-share and HK datasets. Its evaluation reports 62.5% and 61.7% accuracy, while prior tensor-based work learned each stock independently and did not use stock commonalities or address sparsity.
Problem
Existing approaches overlook interactions among heterogeneous sources and correlations among stocks, while event information is sparse and difficult to analyze quantitatively.
Method
The paper jointly models Web-news events, social-media sentiments, quantitative stock features, and stock correlations through coupled matrix and tensor factorization.
Results
62.5% and 61.7% accuracy are reported on the China A-share and HK stock datasets, respectively, demonstrating the model’s effectiveness.
Takeaways & Limitations
The framework predicts correlated stocks simultaneously by sharing collaboratively factorized low-rank matrices across matrices and the tensor.
Takeaways & Limitations
Prior tensor-based frameworks modeled each stock as an independent task, did not exploit stock commonalities, and did not address data sparsity.
Abstract
from arXiv · showhide
Traditional stock market prediction approaches commonly utilize the historical price-related data of the stocks to forecast their future trends. As the Web information grows, recently some works try to explore financial news to improve the prediction. Effective indicators, e.g., the events related to the stocks and the people's sentiments towards the market and stocks, have been proved to play important roles in the stocks' volatility, and are extracted to feed into the prediction models for improving the prediction accuracy. However, a major limitation of previous methods is that the indicators are obtained from only a single source whose reliability might be low, or from several data sources but their interactions and correlations among the multi-sourced data are largely ignored. In this work, we extract the events from Web news and the users' sentiments from social media, and investigate their joint impacts on the stock price movements via a coupled matrix and tensor factorization framework. Specifically, a tensor is firstly constructed to fuse heterogeneous data and capture the intrinsic relations among the events and the investors' sentiments. Due to the sparsity of the tensor, two auxiliary matrices, the stock quantitative feature matrix and the stock correlation matrix, are constructed and incorporated to assist the tensor decomposition. The intuition behind is that stocks that are highly correlated with each other tend to be affected by the same event. Thus, instead of conducting each stock prediction task separately and independently, we predict multiple correlated stocks simultaneously through their commonalities, which are enabled via sharing the collaboratively factorized low rank matrices between matrices and the tensor. Evaluations on the China A-share stock data and the HK stock data in the year 2015 demonstrate the effectiveness of the proposed model.
1. Introduction
Stock prediction research increasingly incorporates Web events and social-media sentiment, but prior approaches often overlook their interactions and the correlations among stocks. This paper addresses these gaps with a coupled matrix and tensor factorization framework evaluated on China A-share and HK stock data.
- Motivation: Web events and users’ sentiments can influence stock prices, while stock-price fluctuations can also stimulate related information and public opinions.The introduction describes this as a two-way relationship between online information and stock movements.
- Challenges: Event-only prediction is limited by sparse, difficult-to-extract information and by differently expressed events that increase sparsity.The paper notes that similar events should be merged into one category to reduce this sparsity.
- Challenges: Events alone do not determine stock impact reliably because similar events can produce positive or negative effects for different companies.The introduction contrasts Microsoft’s acquisition of LinkedIn with Intel’s acquisition of Altera.
- Challenges: Combining heterogeneous sources is difficult because they differ in time scales and structures, while simple feature concatenation assumes independence despite coupling effects.The paper gives event–sentiment relationships and correlations among stocks or industries as examples of such interactions.
- Approach: The proposed framework integrates Web news events, social-media sentiments, quantitative stock information, and stock correlations through coupled matrix and tensor factorization.It models heterogeneous information jointly and supports simultaneous multi-task prediction by sharing factorized representations.
- Evaluation: 62.5% and 61.7% accuracy are achieved on the China A-share and HK stock datasets, respectively, with superiority over the state-of-the-art baseline and fewer parameters to tune.The reported evaluation covers two stock-market datasets and identifies improved performance relative to the baseline.
2. Related work
Prior stock-prediction studies use news events, social-media sentiment, or both, but often omit cross-source interactions, stock relationships, or tensor sparsity. The proposed framework integrates events, sentiments, historical quantitative data, and stock correlations in a coupled matrix and tensor factorization model.
- Event and sentiment signals: News-based studies model how financial events affect stock prices, while sentiment-based studies examine market effects from news or social media.Sentiment signals include news text, public mood, emotional tweets, and sentiment time series.
- Event and sentiment signals: Methods using sentiments often ignore news events, whereas methods using events may overlook investors’ moods.The related work identifies these as complementary information sources.
- Joint modeling limitations: Joint event-sentiment models integrate both signals with firm-specific features, but may construct a separate tensor for each stock and day.This design creates a time-series tensor stream with stock-specific modeling.
- Joint modeling limitations: Per-stock tensor models require many parameters, fail to exploit relationships among stocks, and do not address data sparsity.These limitations hinder robustness and prevent shared stock commonalities from improving individual tasks.
- Proposed direction: This paper integrates events, sentiments, historical quantitative data, and stock correlations through a coupled matrix and tensor framework.It predicts correlated stocks jointly by sharing collaboratively factorized low-rank matrices.
3. Preliminaries
The preliminaries introduce tensor notation, tensor decomposition and reconstruction, and coupled attribute-value similarities. These concepts support representing latent component interactions and modeling both within-attribute and cross-attribute relationships.
- Notation: The paper uses scalars, vectors, matrices, and third-order tensors as progressively higher-order data structures.Third-order tensors are represented as three-dimensional arrays.
- Tensor decomposition and reconstruction: Tensor factorization decomposes data into factor matrices and a core tensor whose entries encode interactions among latent components.Reconstruction multiplies the core tensor with the factor matrices.
- Tensor decomposition and reconstruction: Tensor decomposition and reconstruction update existing entries and fill new entries to reveal latent relationships.The paper presents tensor factorization as an extension of matrix decomposition.
- Coupled attribute value similarity: Coupled attribute-value similarity considers both similarity within an attribute and interactions among different attributes.The information table formalizes instances, attributes, possible values, and attribute-value mappings.
- Coupled attribute value similarity: Inter-coupled similarity aggregates relative similarities based on other attributes, while intra-coupled similarity uses attribute-value frequency distributions.The relative similarity between two values of one attribute is conditioned on another attribute.
4. The system framework
The system combines historical quantitative stock data, Web news, and social media into a tensor and auxiliary matrices for stock price movement prediction. It constructs stock features and correlations, extracts events and sentiments, and uses coupled stock relationships to support prediction.
- The framework uses historical stock quantitative data, Web news, and social media to model joint effects on stock price movements.
- The system has four stages: quantitative feature construction, stock correlation construction, event and sentiment extraction, and coupled factorization for prediction.
- Building the stock movement tensor: The stock movement tensor is built from extracted events and sentiment polarities, then reconstructed tensor entries are used for stock predictions.
- Building the stock quantitative feature matrix: The quantitative feature matrix uses normalized share turnover, P/E, P/B, and PCF features to represent each stock comprehensively.
- Building the stock correlation matrix: The stock correlation matrix incorporates coupled effects among features, including closing prices and industry index trends.
- Building the stock correlation matrix: Coupled Stock Similarity compares stock attributes using intra-attribute and inter-attribute interactions, with daily values averaged over a nine-month training period.
5. Coupled matrix and tensor factorization
The method collaboratively factorizes a sparse stock movement tensor with quantitative-feature and stock-correlation matrices. Shared low-rank factors allow multiple correlated stock-prediction tasks to be co-learned rather than treated independently.
- The model shares low-rank matrices across X, Y, and Z, propagating information through collective matrix and tensor factorization.
- Multiple stock-prediction tasks are co-learned simultaneously through commonalities and shared knowledge instead of being conducted independently.
- A sparse stock movement tensor is completed by jointly decomposing it with the quantitative feature matrix X and stock correlation matrix Z.
- Tensor A is decomposed into a core tensor and low-rank factors for stocks, events, and sentiments, while X is factorized as X = U × F.
- The objective minimizes tensor and matrix factorization errors, applies stock-correlation Laplacian regularization, and penalizes factor norms to avoid overfitting.
- The correlation penalty encourages highly correlated stocks to have closer latent stock vectors in U.
- Because the objective is not jointly convex, the method updates variables element-wise and iteratively using gradient descent.
6. Experiments
The experiments evaluate CMT on China A-share and HK stock datasets using heterogeneous market, news, and social-media information. CMT achieves strong movement-prediction performance, while ablations and correlation variants support the value of auxiliary matrices and stock-correlation modeling.
- Comparison methods: CMT jointly factorizes two auxiliary matrices and a tensor, whereas comparison methods include SVM, PCA+SVM, TeSIA, and CMT variants omitting one or both auxiliary matrices.The CMT-Z and CMT-Z-X variants test the contributions of the stock-correlation matrix and both auxiliary matrices, respectively.
- Prediction results: 62.5% ACC and 0.409 MCC are reported for CMT on the China stock market, where it achieves the best performance among the compared methods.SVM performs worst, while PCA+SVM improves over SVM; CMT also outperforms CMT-Z, indicating the importance of stock-correlation information in these experiments.
- Prediction results: CMT improves accuracy by 3% and MCC by 115% compared with TeSIA, and the HK-market results are reported as similar to those on A-share data.The authors also report that CMT requires fewer parameters to tune than TeSIA, whose tuning requirements grow linearly with the number of stocks.
- Correlation analysis: The coupled stock-correlation method outperforms the other tested correlation methods on both datasets in ACC and MCC.The authors attribute this result to more realistic correlations obtained by considering coupled effects between stock attributes.
7. Conclusions and future work
The paper concludes that coupled matrix and tensor factorization integrates heterogeneous stock information and supports prediction from a sparse movement tensor. Evaluations on China A-share and HK data show effectiveness, while future work targets richer data, improved event and sentiment processing, and time-series extensions.
- Conclusions: The proposed framework integrates events, sentiments, and quantitative features from Web news, social media, and a stock-data provider.A stock movement tensor models relations among stocks, events, and sentiments.
- Conclusions: Coupled matrix and tensor factorization addresses sparse event information by learning stock correlations and collaboratively factorizing the tensor with two auxiliary matrices.The resulting low-rank matrices support movement prediction by completing missing values in the sparse tensor.
- Conclusions: Evaluations on China A-share and HK stock data demonstrate the effectiveness of the proposed model.
- Future work: Future work includes adding knowledge-graph information, using domain knowledge for event representations, and developing more accurate domain-specific sentiment classification.The authors also identify extending the model to a time-series prediction framework as an open direction.
- Future work: The framework is motivated by evidence that psychological biases such as overconfidence affect financial-market decision-making and may be incorporated into prediction.