Source-linked AI summary
Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction
Jonathan Chang, Zimeng Lyu
TL;DR
Return forecasts drive allocation, yet forecast accuracy alone does not establish whether a model supports profitable trading decisions. The paper compares model families using pooled stock panels and forecast and strategy measures; EXAMM ranks first across forecast and trading metrics in the evaluated portfolios and years.
Problem
Return forecasts feed portfolio allocation, but open questions include whether forecasting models support trading decisions and outperform buy-and-hold, and whether their computational costs are warranted.
Source: §1 · p. 1Method
The study compares linear, fixed recurrent, transformer, mixing-based, and EXAMM-evolved recurrent models trained on pooled panels using forecast metrics and a daily long/short strategy.
Source: §1 · p. 1Results
Across four disjoint portfolios and three trading years, EXAMM ranks first on IC, net return, Sharpe, and MDD; IC is +0.0106 versus PatchTST’s +0.0069.
Source: §6.5 · p. 6, §7 · p. 7Takeaways & Limitations
PatchTST’s second-highest IC (+0.0069) accompanies a −1.55% net return, showing that strong cross-sectional ranking need not yield profitable trades.
Source: §6.5 · p. 6The validation construction projects current S&P 500 constituents with 20 years of continuous history backward, so reported returns should be read as relative rather than absolute levels.
Source: §4.2 · p. 3
Abstract
from arXiv · showhide
Time series forecasting models are typically compared on pointwise error, which scores a prediction in isolation from the decision it is produced for, and a lower forecast error does not imply a better decision downstream. A parallel debate asks whether modern transformer architectures forecast better than recurrent and other lightweight models. We compare linear, fixed recurrent, transformer, and mixing based architectures against recurrent networks evolved by neuroevolutionary architecture search, evaluating each on forecast accuracy and on the net return of a daily long/short strategy. All models are fit on a pooled panel, one network trained across the whole universe. Across four mid-cap portfolios and three trading years, the evolved networks rank first on both forecast accuracy and net trading performance, while the second most accurate model loses money once positions are formed and costs are charged. The advantage tracks a horizon match, since rank IC for the evolved networks rises from a one-day to a ten-day scoring horizon while every model above 300 parameters declines. They are also the cheapest end to end: a CPU-only search of 16 minutes yields 66-weight networks that predict in 10.8~$\mu$s on a Raspberry Pi Zero, against transformer baselines of up to 817,153 parameters that require GPU training.
1 Introduction
Return forecasts feed portfolio allocation and trading, but forecasting benchmarks score predictions apart from the decisions they inform. The paper compares model families on forecast accuracy and trading outcomes while examining panel construction and computational cost.
1 Introduction
- Forecasting architectures remain contested, and pointwise error does not establish whether forecasts improve the trading decisions they inform.The comparison includes recurrent, transformer, and lightweight alternatives.Source: §1 · p. 1
- Three open questions concern training one model across stocks, beating buy-and-hold in trading, and whether computational costs are warranted.Source: §1 · p. 1
- The study compares linear, fixed recurrent, transformer, mixing-based, and EXAMM-evolved recurrent models on pooled stock data and a daily long/short strategy charged half-spread costs.Evaluation covers information coefficient, net return, Sharpe ratio, and maximum drawdown across four disjoint mid-cap portfolios and three trading years.Source: §1 · p. 1
- The paper contributes a pooled-panel construction and a controlled comparison that assesses forecast accuracy alongside realized trading performance.It also evaluates end-to-end computational cost and reports that the smallest models are the strongest.Source: §1 · pp. 1–2
2 Related Work
Prior work spans statistical, machine-learning, and deep architectures for financial forecasting, while debate continues over the value of added model capacity. Related research also links forecasts to portfolio decisions and applies neuroevolution to recurrent-network design.
2 Related Work
- Financial forecasting research spans ARIMA, nonlinear machine-learning models, recurrent, convolutional, graph, attention, state-space, language, and diffusion approaches.Whether added capacity is necessary remains contested, with DLinear reported to match or exceed transformers on standard benchmarks [44].Source: §2 · p. 2
- In cross-sectional forecasting, allocation consumes forecasts, and decision-focused objectives can improve realized portfolio performance despite raising prediction error [18].The compared architectures include linear, patch-based, variate-tokenizing, deformable-attention, and multiscale MLP models.Source: §2 · p. 2
- Neuroevolution searches network structure while gradient descent trains candidate weights, and EXAMM evolves recurrent topologies, cell types, and deep connections [30].Prior equity-forecasting work reports EXAMM networks competitive with transformers at a fraction of their parameter count.Source: §2 · p. 2
3 Method
The method uses a pooled panel to train one model across stocks, evolves recurrent architectures with EXAMM, and evaluates predictions through a gated daily long/short strategy.
3.1 Panel Construction
- Pooled training fits one model to the union of stocks’ examples while retaining d predictors and one output, unlike per-stock or wide concatenated arrangements.The input dimension stays fixed as the number of stocks grows, while training examples grow in proportion to that number.Source: §3.1 · p. 2
3.2 EXAMM
- EXAMM evolves candidate recurrent networks through selection, mutation, and crossover, training each child before retaining it when its validation error beats the population’s worst member.Search begins from minimal input-to-output networks and can add nodes, edges, recurrent delays, or memory-cell types.Source: §3.2 · p. 2
- EXAMM can mix recurrent cell types within a network and evolve deep recurrent connections that access past lags directly.Its asynchronous workers train genomes independently, while islands maintain structural diversity through periodic exchange and repopulation.Source: §3.2 · pp. 2–3
3.3 Daily Long–Short Trading Strategy
- The daily strategy sorts stocks by predicted return and rebalances only when all selected top-ranked predictions are positive and all selected bottom-ranked predictions are negative.On rebalance, it liquidates existing positions and reallocates capital equally within long and short sides; otherwise, it holds positions.Source: §3.3 · p. 3
4 CRSP Dataset
The study derives daily-frequency equity data and six predictors from CRSP, using a baseline 50-stock portfolio and four additional validation portfolios. Walk-forward folds test the portfolios across different years and market conditions.
4 CRSP Dataset
- The study uses daily-frequency data derived from CRSP and six predictors: return, volume change, bid-ask spread, illiquidity, turnover, and S&P 500 return.Source: §4 · p. 3
4.1 Baseline Portfolio
- The baseline portfolio follows Lyu et al.’s selection of 50 mid-cap S&P 500 companies, retaining the same tickers for direct comparison.The portfolio uses the six predictors obtained from or computed from CRSP variables.Source: §4.1 · p. 3
4.2 Validation Portfolios
- Four disjoint validation portfolios contain 200 randomly selected securities drawn from a mid-cap to low-large-cap universe with at least 20 years of continuous trading history.The current-constituent and history requirements create survivorship bias, so reported returns should be read as relative rather than absolute levels.Source: §4.2 · p. 3
4.3 Pooled Panel and Walk-Forward Evaluation
- Baseline evaluation uses two walk-forward folds testing 2022 and 2023, while validation portfolios use three folds testing 2022–2024.The baseline test years span the 2022 bear market and 2023 bull recovery; the validation design extends evaluation across stock selection and market conditions.Source: §4.3 · pp. 3–4, Table 1 · p. 4, Table 2 · p. 4
5 Experimental Setup
The study compares fixed and evolved forecasting models under a shared next-day target, then evaluates predictions through a daily long-short strategy and benchmark-adjusted returns.
5.1 Models
- EXAMM evolves recurrent topologies using validation MSE, while fixed baselines include linear, transformer, mixing-based, and two-layer recurrent models.The fixed models use an input length of 96 and output length of 1; the recurrent baselines train for 1,000 epochs, and configurations are repeated with 10 seeds.Source: §5.1 · p. 4
5.2 Trading Strategy and Costs
- Each day, the strategy ranks predicted returns and takes equal-weight long and short positions in the top and bottom 10 stocks when its entry condition is met.Every trade is charged the stock-specific half quoted spread, which is subtracted from reported net returns.Source: §5.2 · p. 4
5.3 Factor Benchmarks
- Daily strategy returns are regressed on the panel’s equal-weight market return and one- and five-day reversal portfolios to distinguish forecast alpha from reversal exposure.The reversal portfolios use the same 10/10 book as the strategy.Source: §5.3 · p. 4
5.4 Computing Environment
- EXAMM runs on CPU with MPI parallelization, while the baselines are trained in PyTorch on NVIDIA RTX 4090 GPUs.Each EXAMM run used 32 cores on shared AMD EPYC nodes.Source: §5.4 · p. 4
6 Results
Across the tested portfolios, EXAMM leads on forecast and trading measures, and its risk-adjusted performance remains positive after reversal controls and trading costs. Its small evolved networks are also inexpensive to search and deploy.
6.1 Pooled vs. Alternative Constructions
- Pooling improves every reported metric in both folds; in 2023, the trading result shifts from a −4.17% loss under individual training to a +30.24% gain.Concatenated construction is worst on every metric; joining series creates boundary discontinuities that recurrent state cannot distinguish from genuine transitions.Source: §6.1 · pp. 4–5
6.2 Forecast Quality
- +0.0106 mean Pearson IC gives EXAMM the highest raw-prediction score, ahead of PatchTST (+0.0069) and DLinear (+0.0067).No model is positive in every panel-year cell.Source: Table 4 · p. 5, §6.2 · p. 5
6.4 Risk-Adjusted Performance
- +10.64% annualized alpha is distinguishable from zero after reversal controls, with a 95% interval of [1.0%, 20.3%] under the fullest specification.Adding reversal controls attenuates alpha from +11.27% to +10.64%; EXAMM’s MKT alpha falls from +11.71% gross to +11.27% net, a 0.44-point annual drag.Source: Figure 1 · p. 6, §6.4 · p. 6
6.3 Portfolio Returns across Market Regimes
- +11.02% is EXAMM’s highest three-year mean net return, versus +2.64% for buy-and-hold and +4.09% for TimeMixer.EXAMM has the highest average return in each of the three trade years; the authors characterize its advantage as consistency across regimes.Source: §6.3 · pp. 5–6, Table 7 · p. 6
6.5 Overall Comparison
- +0.0106 IC, +11.02% net return, +0.74 Sharpe, and −10.76% MDD make EXAMM best on all four Table 8 metrics.PatchTST has the second-highest IC (+0.0069) but a net return of −1.55%.Source: §6.5 · p. 6, Table 8 · p. 7
- +0.0068 to +0.0142: EXAMM’s rank IC rises from a one-day to a ten-day horizon, while every model above 300 parameters declines.The entry rule leads EXAMM to hold positions for several days, matching its stronger forecast horizon to the period over which it trades.Source: §6.5 · p. 6, Figure 2 · p. 7
6.6 Scalability and Deployment
- 16.2 minutes at 32 ranks completes the CPU-only search; the resulting 66-parameter networks predict in 10.82 μs on a Raspberry Pi Zero.The search is asynchronous and scales near-linearly with workers; transformer baselines require GPU training and reach 817,153 parameters.Source: §6.6 · p. 7, Table 9 · p. 7
7 Conclusion
Across four disjoint portfolios and three trading years, EXAMM ranks first on information coefficient, net return, Sharpe ratio, and maximum drawdown. The conclusion links this performance to pooled-panel cross-sectional ranking, while reporting positive alpha that survives reversal controls and low computational cost.
7 Conclusion
- Across four disjoint portfolios and three trading years, EXAMM ranks first on information coefficient, net return, Sharpe ratio, and maximum drawdown.It also remains profitable during the 2022 drawdown and the 2023 and 2024 recoveries.Source: §7 · p. 7
- Pooling improves cross-sectional ranking rather than pointwise accuracy: relative to per-stock training, MSE falls by less than 4% while information coefficient turns positive in both folds.The information coefficient is the quantity consumed by a rank-based allocation.Source: §7 · p. 7
- EXAMM is the only model with alpha positively distinguishable from zero after short-term reversal controls, and its evolved networks average 66 weights.The networks are searched on CPU in under twenty minutes on 32 cores and evaluate in 10.82 μs on a Raspberry Pi Zero; parameter count is unrelated to realized return across the eight models.Source: §7 · p. 7