Source-linked AI summary
AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks
Jingyuan Wang, Yang Zhang, Ke Tang, Junjie Wu, Zhang Xiong
TL;DR
Deep-learning quantitative-trading strategies face unresolved concerns about risk-return balance, extreme-loss resistance, and interpretability. AlphaStock addresses these concerns with reinforcement learning and interpretable deep attention networks, and experiments report robust performance across U.S. and Chinese markets while revealing a specific winner-selection pattern.
Problem
Existing deep-learning investment strategies face unresolved challenges in balancing risk and return, modeling asset interrelationships, controlling extreme losses, and interpreting decisions.
Method
AlphaStock combines reinforcement learning, LSTM history-state attention, cross-asset attention, a portfolio generator, Sharpe-ratio optimization, and sensitivity analysis for an interpretable BWSL strategy.
Results
Experiments report that AlphaStock outperforms competing strategies across evaluation measures and is effective and robust across U.S. and Chinese markets.
Takeaways & Limitations
AlphaStock selects winners with high long-term growth, low volatility, high intrinsic value, and recent undervaluation.
Takeaways & Limitations
The paper uses a simplifying approximation and ignores real-world transaction costs.
Abstract
from arXiv · showhide
Recent years have witnessed the successful marriage of finance innovations and AI techniques in various finance applications including quantitative trading (QT). Despite great research efforts devoted to leveraging deep learning (DL) methods for building better QT strategies, existing studies still face serious challenges especially from the side of finance, such as the balance of risk and return, the resistance to extreme loss, and the interpretability of strategies, which limit the application of DL-based strategies in real-life financial markets. In this work, we propose AlphaStock, a novel reinforcement learning (RL) based investment strategy enhanced by interpretable deep attention networks, to address the above challenges. Our main contributions are summarized as follows: i) We integrate deep attention networks with a Sharpe ratio-oriented reinforcement learning framework to achieve a risk-return balanced investment strategy; ii) We suggest modeling interrelationships among assets to avoid selection bias and develop a cross-asset attention mechanism; iii) To our best knowledge, this work is among the first to offer an interpretable investment strategy using deep reinforcement learning models. The experiments on long-periodic U.S. and Chinese markets demonstrate the effectiveness and robustness of AlphaStock over diverse market states. It turns out that AlphaStock tends to select the stocks as winners with high long-term growth, low volatility, high intrinsic value, and being undervalued recently.
1 INTRODUCTION
AlphaStock addresses three challenges in deep-learning quantitative trading: balancing risk and return, modeling relationships among assets, and interpreting investment decisions. It combines reinforcement learning with deep attention networks and shows strong performance across U.S. and Chinese markets.
- 1 INTRODUCTION: The paper identifies limited asset-interrelationship modeling as a gap because existing deep-learning and reinforcement-learning strategies largely overlook information used in hedging, arbitrage, and BWSL strategies.
- 1 INTRODUCTION: The paper treats interpretability as an open problem because deep reinforcement-learning strategies can become especially difficult to explain.
- 1 INTRODUCTION: AlphaStock combines deep attention networks with Sharpe-ratio-oriented reinforcement learning to pursue risk-return-balanced investment decisions.The framework is designed to integrate representation learning with risk-aware optimization.
- 1 INTRODUCTION: AlphaStock uses a cross-asset attention network to model asset relationships and a portfolio generator to assign investment proportions from winner scores.
- 1 INTRODUCTION: Experiments on long-periodic U.S. and Chinese markets report that AlphaStock outperforms state-of-the-art competitors across evaluation measures and adapts to diverse market states.The reported strategy also controls extreme losses and selects winners with high long-term growth, low volatility, high intrinsic value, and recent undervaluation.
2 PRELIMINARIES
The preliminaries define sequential long-short investment and the BWSL strategy, in which winners are bought and losers are sold. They formulate performance through return, volatility, and a Sharpe-ratio objective that balances profit and risk over time.
- 2.1 Basic Financial Concepts: Sequential investment carries capital and profits from one holding period into the next, while asset prices are represented as time series over those periods.
- 2.1 Basic Financial Concepts: Long positions buy assets before selling them, whereas short positions sell borrowed assets before buying them back and returning them to the broker.
- 2.1 Basic Financial Concepts: A portfolio is a vector of asset investment proportions, and a zero-investment portfolio combines positions whose net total investment is zero.The paper notes a budget constraint on borrowed stocks and ignores real-world transaction costs for simplicity.
- 2.2 The BWSL Strategy: BWSL forms a zero-investment portfolio with a long portfolio for buying high-rising-rate winners and a short portfolio for selling low-rising-rate losers.The strategy can profit when long-portfolio stocks outperform short-portfolio stocks even if the overall market falls.
- 2.3 Optimization Objective: The framework motivates steady investment by prioritizing profit and risk together rather than maximizing short-term returns alone.
- 2.3 Optimization Objective: The Sharpe ratio evaluates average excess return per unit of volatility, so the objective maximizes long- and short-portfolio sequences while accounting for long-term return variability.The formulation includes transaction costs in average return and defines volatility from period returns.
3 THE ALPHASTOCK MODEL
AlphaStock combines sequential and history-state attention, cross-asset attention, and Sharpe-ratio-optimized reinforcement learning to generate winner and loser portfolios. Its features cover trading and company information, while CAAN models stock interrelationships and price-rising-rank priors.
- 3 THE ALPHASTOCK MODEL: AlphaStock uses LSTM-HA to encode each stock’s history, CAAN to model interrelationships, and a portfolio generator to allocate winner and loser positions.The three components are optimized end-to-end by maximizing the Sharpe ratio over sequential investments.
- 3.1 Raw Stock Features: The model represents stocks using trading features and company features, including price rising rate, volatility, volume, dividends, market capitalization, price-earnings ratio, and book-to-market ratio.The feature values are standardized into Z-scores before use.
- 3.2 Stock Representations Extraction: LSTM-HA combines sequential dependence from an LSTM with global and long-range dependence from history-state attention.Shared parameters extract representations for all stocks, supporting relatively stable and general representations.
- 3.3 Winners and Losers Selection: CAAN applies self-attention to stock representations, using queries, keys, and values to weight other stocks’ information when estimating winner scores.The normalized interrelationships determine how value vectors from other stocks contribute to each stock’s attention vector.
- 3.3 Winners and Losers Selection: CAAN can incorporate prior information from the relative price-rising ranks of stocks through discretized distances and learnable relation embeddings.This prior augments interrelationships learned directly from data.
4 MODEL INTERPRETATION
AlphaStock interprets winner selection by measuring how individual historical stock features influence the model’s winner score. The sensitivity measure is aggregated across stock states to identify features associated with higher or lower winner scores.
- 4 MODEL INTERPRETATION: The interpretation method measures each history feature’s influence on a stock’s winner score within the combined LSTM-HA and CAAN network.The function s = F(X) maps a stock’s history features X to winner score s.
- 4 MODEL INTERPRETATION: The average influence of a feature is estimated across possible stock states and approximated over stocks and holding periods in the dataset.The formulation uses the probability density of stock history states and a dataset-level average.
- 4 MODEL INTERPRETATION: A positive average influence indicates that larger feature values tend to increase winner scores, while a negative influence indicates the opposite.The reported negative influence for fine-grained volatility means the model tends to select lower-volatility stocks as winners.
5 EXPERIMENT
AlphaStock is evaluated against traditional, reinforcement-learning, and ablated strategies on U.S. and Chinese markets using return, risk, and interpretability analyses. It generally performs better than the baselines, remains stable across market states, controls extreme losses, and yields interpretable stock-selection patterns.
- 5.4 Performance in U.S. Markets: AlphaStock outperforms the baselines across cumulative wealth and the reported evaluation measures in U.S. markets.The comparison includes Market, momentum, reversion, FDDR, and AlphaStock ablations.
- 5.4 Performance in U.S. Markets: AlphaStock performs more stably across diverse market states than traditional strategies, which excel in only particular bull or bear conditions.TSM performs well in bull markets but poorly in bear markets, whereas RMR shows the opposite pattern.
- 5.4 Performance in U.S. Markets: AlphaStock, AlphaStock-NP, and AlphaStock-NC outperform other baselines on all reported measures, while AlphaStock and AlphaStock-NP achieve higher APR than AlphaStock-NC at slightly higher volatility.The reported measures include AVOL, MDD, ASR, CR, and DDR.
- 5.4 Performance in U.S. Markets: AlphaStock’s extreme-loss control ranks AlphaStock > AlphaStock-NP > AlphaStock-NC on MDD, CR, and DDR, highlighting the contributions of CAAN and the price-rising rank prior.The authors connect these components to exploiting stock-ranking relationships and avoiding buying losers while selling winners.
- 5.5 Performance in Chinese Markets: On Chinese markets, AlphaStock and its ablations again outperform other baselines, while their AVOL and MDD remain higher than in U.S. markets but competitive relative to baselines.The authors attribute the higher Chinese-market risk partly to market conditions and the lack of short selling.
- 5.6 Investment Strategies Interpretation: Interpretation analyses indicate that AlphaStock favors stocks with high long-term growth, low volatility, high intrinsic value, and recent undervaluation.Long-term and short-term price-rising effects differ, while valuation features such as MC, PE, and BM have positive influence on winner scores.
6 RELATED WORKS
Prior work spans classic financial strategies, deep learning for financial prediction, and reinforcement learning for investment, but each has limitations in complex markets or portfolio generation.
- Financial Investment Strategy: Classic momentum, mean reversion, and multi-factor strategies exploit different market signals but may fail in complex market environments.Momentum selects assets by past returns, mean reversion by deviation from historical means, and multi-factor models by estimated valuations.
- Deep Learning in Finance: Deep learning has been applied to price prediction using frequency, news, social-media, event, and corporate-relationship information.These studies generally focus on predicting prices rather than generating investment portfolios end to end.
- Reinforcement Learning in Finance: Reinforcement-learning investment methods include value-based and policy-based approaches, with value-based methods limited by the complexity of approximating market outcomes with a critic.Representative value-based strategies include Q-learning and deep Q-learning.
7 CONCLUSIONS
The paper proposes AlphaStock, a reinforcement-learning deep-attention BWSL strategy that models stock interrelationships and interprets investment logic. Experiments in U.S. and Chinese markets report stronger performance than competing strategies and identify characteristics associated with selected winners.
- Conclusions: AlphaStock performed much better than competing strategies in back-testing and simulation experiments across U.S. and Chinese stock markets.The conclusion reports this outcome across both evaluated markets.
- Conclusions: AlphaStock is a reinforcement-learning deep-attention network for buying winners and selling losers that exploits interrelationships among stocks.The paper also presents a sensitivity analysis method for interpreting its investment logic.
- Conclusions: AlphaStock suggests selecting winners with high long-term growth, low volatility, high intrinsic value, and recent undervaluation.These characteristics summarize the investment patterns identified by the model’s interpretation analysis.