Source-linked AI summary
AlphaRJM: Reward-Jump Memory for Stochastic Return-Guided Alpha Discovery
Sayan Dhan, Selvaraju Natarajan
TL;DR
Formulaic alpha discovery is pool-dependent, and terminal feedback leaves both evaluation history and intermediate action value uncertain. AlphaRJM combines persistent terminal-feedback memory with a stochastic return critic, and reports strong, stable performance across tested universes and horizons.
Problem
Pool-dependent alpha search receives informative feedback primarily at complete-expression evaluations, while the retained pool does not preserve the full history of realized feedback.
Source: §1 · p. 1Method
AlphaRJM combines event-driven Reward-Jump Memory, which updates from terminal rewards and outcomes, with an action-conditioned SDE critic whose return particles guide action selection.
Source: §1 · p. 2Results
Across CSI300, CSI500, and CSI800, AlphaRJM shows strong, stable correlation-based performance against conventional, continuous-time, and formulaic baselines, including at a 42-trading-day horizon.
Source: §5 · p. 9Takeaways & Limitations
The results indicate that preserving terminal reward–outcome history and modeling future discovery returns distributionally provide complementary signals for pool-dependent symbolic alpha discovery.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Formulaic alpha discovery is a pool-dependent symbolic search problem in which informative feedback is observed primarily when a complete expression is evaluated. This delayed feedback creates two coupled difficulties: the retained alpha pool does not preserve the full history of realized evaluation feedback, and the value of an intermediate construction action is uncertain because its consequence depends on the formula eventually completed. We introduce AlphaRJM, which addresses these difficulties through Reward-Jump Memory, an event-driven latent state that remains fixed during token construction and updates only at terminal evaluation events using the realized pool reward and evaluation outcome, and an action-conditioned SDE return critic that represents future discounted discovery returns with stochastic particles. The particles guide action selection through their mean and uncertainty and are learned using a distributional Bellman objective combining energy-distance matching, mean calibration, and jump regularization. Empirically, AlphaRJM delivers strong and stable gains across multiple equity universes, forecasting horizons, and random seeds, while ablations confirm the complementary roles of persistent evaluation history, stochastic return modeling, and distributional supervision.
1 INTRODUCTION
Formulaic alpha discovery is pool-dependent and receives sparse, evolving feedback at terminal evaluations. AlphaRJM combines persistent terminal-feedback memory with stochastic return-guided action selection, and reports strong, stable gains across tested settings.
Motivation
- A formula’s value depends on its contribution to the existing alpha pool, so stronger but redundant candidates may be less useful than complementary ones.This pool-dependent search receives sparse, evolving feedback.Source: §1 · p. 1
- Different terminal outcomes can produce distinct evaluation histories even when the retained pool is unchanged, so pool contents alone omit search feedback.Rejected, accepted or replaced, and invalid candidates yield distinct terminal events.Source: §1 · p. 1
Approach
- Reward-Jump Memory preserves realized pool rewards and categorical outcomes across formula episodes, remaining fixed during token construction and updating only after terminal evaluation.The memory carries evaluation history independently of whether the retained pool changes.Source: §1 · p. 2
- The action-conditioned SDE critic represents uncertain future discovery returns with particles whose mean and uncertainty guide symbolic action selection.An intermediate action’s utility depends on the expression eventually completed and the pool used for evaluation.Source: §1 · p. 2
Empirical results
- AlphaRJM achieves strong, stable gains across multiple equity universes, forecasting horizons, and random seeds, with ablations supporting the complementary roles of its components.The reported evidence also supports persistent evaluation history, stochastic return modeling, and distributional supervision as complementary contributions.Source: §1 · p. 2
Training
- The framework uses distributional Bellman learning with sequence-consistent memory replay to train its return critic.Its effectiveness is evaluated across multiple equity universes, forecasting horizons, and random seeds.Source: §1 · p. 2
2 BACKGROUND AND RELATED WORK
Prior work addresses pool-dependent rewards, distributional exploration, and symbolic representation through different methods. AlphaRJM’s distinct emphasis is retaining terminal evaluation history and using it to condition future return evaluation.
Pool-dependent rewards
- AlphaGen formalizes pool dependence by rewarding a candidate for its marginal contribution to the combined alpha ensemble (Yu et al., 2023).AlphaRJM adopts this setting while adding persistent terminal-feedback history across formula episodes.Source: §2 · p. 2
Related approaches
- AlphaRJM conditions particle-based future-return distributions on the current expression, pool, persistent evaluation history, and candidate action.This extends the distributional perspective used in prior return modeling, including AlphaQCM (Bellemare et al., 2017; Dabney et al., 2018).Source: §2 · p. 2
AlphaRJM’s distinction
- AlphaRJM couples persistent terminal reward–outcome information with a sampleable distribution of future discovery returns, addressing a different search aspect from AlphaSAGE’s representation and exploration approach.Its SDE serves solely as an internal stochastic return critic, not as a model of asset prices or alpha signals.Source: §2 · p. 3
3 METHODOLOGY
AlphaRJM augments grammar-based symbolic search with persistent memory of terminal evaluation feedback and an action-conditioned stochastic return critic. Its methodology covers the search state, event-driven memory, return-guided action selection, and distributional Bellman training.
3 METHODOLOGY
- The methodology couples event-driven memory of terminal feedback with stochastic return-guided symbolic search.It formalizes the search state and terminal events before introducing memory, action selection, and distributional Bellman learning.Source: §3 · p. 3
3.1 FRAMEWORK AND SEARCH STATE
- At each construction step, the search state represents the partial expression and retained pool, while terminal events end formula episodes and yield a reward based on the pool’s marginal training-set ensemble IC change.The same candidate can have different utility under different retained pools.Source: §3.1 · pp. 3–4
3.2 REWARD-JUMP MEMORY
- Reward-Jump Memory encodes terminal rewards and outcomes into bounded state jumps, stays constant during token construction, and persists across formula boundaries.The memory stream continues when a formula episode ends, even though the Bellman episode does not.Source: §3.2 · p. 4
3.3 STOCHASTIC RETURN-GUIDED ALPHA DISCOVERY
- The action-conditioned SDE critic generates particles for uncertain discounted returns, using their empirical mean and variance in action scoring with an uncertainty emphasis that decreases over search.The return process is a sampleable scalar model conditioned on search context and candidate action.Source: §3.3 · p. 5
3.4 DISTRIBUTIONAL BELLMAN LEARNING
- Distributional Bellman learning matches predicted and target return distributions, calibrates the mean, regularizes memory jumps, and trains with contiguous replay sequences to preserve memory trajectories.At a terminal event, the Bellman target reduces to the realized pool reward without bootstrapping across formula boundaries, while memory persists.Source: §3.4 · pp. 5–6
4 EXPERIMENTS AND RESULTS
AlphaRJM is evaluated across three equity universes, multiple forecasting horizons, and random seeds, with overall performance, long-horizon generalization, and sensitivity all examined.
4.1 EXPERIMENT SETTING
- The experiments use IC, ICIR, RIC, and RICIR on CSI300, CSI500, and CSI800, with a common chronological split and conventional, continuous-time, and formulaic-alpha baselines.The standard forecasting horizon is h = 20, while the long-horizon experiment uses h = 42.Source: §4.1 · p. 6
4.2 OVERALL PERFORMANCE AND ROBUSTNESS
- AlphaRJM achieves the best mean result in 10 of 12 dataset–metric comparisons and ranks second in the remaining two across CSI300, CSI500, and CSI800.It leads both IC and ICIR on all three universes, and all four metrics on CSI300 and CSI500.Source: §4.2 · p. 7, Table 1 · p. 7
- On CSI500, AlphaRJM maintains the strongest cumulative-wealth trajectory for most of the 2022–2024 test period and recovers strongly after major declines.The CSI500 index remains substantially below the learned strategies over most of the evaluation period.Source: §4.2 · pp. 7–8, Figure 1 · p. 8
4.3 LONG-HORIZON GENERALIZATION
- At h = 42, using random seed 0, AlphaRJM ranks first on IC and RIC for CSI300 and CSI500, scoring (0.046, 0.066) and (0.051, 0.065), respectively.This covers all four long-horizon dataset–metric comparisons.Source: Table 2 · p. 7, §4.3 · p. 8
4.4 SENSITIVITY ANALYSIS
- Sensitivity tests show fewer SDE particles generally yield stronger correlation metrics, while performance is relatively insensitive to integration steps and excessive diffusion degrades performance.The energy-distance weight trades off IC- and rank-based metrics, and moderate σmax gives the most balanced results; behavior is smooth and stable across tested configurations.Source: §4.4 · p. 8
5 CONCLUSION
The conclusion presents AlphaRJM as coupling persistent terminal-evaluation history with stochastic return-guided search, and reports gains across universes and horizons. The appendices describe LLM use, implementation scope, and the search loop.
5 CONCLUSION
- Across CSI300, CSI500, and CSI800, AlphaRJM achieves strong and stable correlation-based performance relative to conventional, continuous-time, and formulaic alpha-discovery baselines.The improvement extends to the 42-trading-day forecasting setting, with ablations supporting complementary contributions from persistent evaluation history, stochastic return modeling, and distributional supervision.Source: §5 · pp. 8–9
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
- LLMs were used only for language refinement, grammatical correction, presentation, and limited assistance with selected code writing and debugging; the authors independently conducted and verified the research.Source: A THE USE OF LARGE LANGUAGE MODELS… · p. 11
B ALPHARJM IMPLEMENTATION DETAILS
- The implementation appendix gives architectural dimensions, symbolic-construction details, numerical SDE integration, exploration schedules, replay, and optimization settings.Source: B ALPHARJM IMPLEMENTATION DETAILS · p. 11
B.1 ARCHITECTURE OVERVIEW
- The search loop encodes the partial expression and retained pool for exploitation, samples from available actions for exploration, and updates memory after terminal formula evaluations.Each terminal evaluation yields a pool reward and outcome before the next formula episode begins.Source: §B.1 · p. 11
B.2 SYMBOLIC CONSTRUCTION AND POOL EVALUATION
The implementation completes formula attempts through a terminal separator, evaluates them against the retained alpha pool, and uses encoded expressions and pool contents as search representations.
B.2 SYMBOLIC CONSTRUCTION AND POOL EVALUATION
- The selectable vocabulary contains 62 actions, and SEP completes the current formula attempt when selected.Source: §B.2 · p. 11
- The available action set applies expression-builder legality rules and a length-dependent early-stop mask, with SEP made available unconditionally at maximum length.The reported runs use pmask = 1, and the mask is cached so action selection and environment execution share the same realization.Source: §B.2 · p. 12
- A terminal action evaluates the completed expression on training data; invalid attempts receive zero reward, while valid candidates are considered for a capacity-50 pool.The reward is the marginal improvement in weighted-ensemble training IC Q(P).Source: §B.2 · p. 12
- When the pool is full, candidates are jointly reweighted with existing members and may be rejected or replace an existing member; accepted or replaced candidates can have zero incremental reward.A candidate receiving the smallest absolute coefficient is rejected and the pre-candidate pool is restored.Source: §B.2 · p. 12
B.3 EXPRESSION AND POOL ENCODERS
- The partial-expression encoder uses 32-dimensional token and positional embeddings with a GRU, while completed formulas reuse that encoder and are projected to 24-dimensional embeddings.Pool-item representations concatenate formula embeddings with coefficient, IC, length, redundancy, and occupancy features; summed item representations provide a permutation-invariant pool aggregate.Source: §B.3 · p. 12
B.4 REWARD-JUMP MEMORY ARCHITECTURE
The appendix details how persistent memory, the stochastic return critic, exploration, replay, and universe-specific settings are implemented. Memory changes only at terminal evaluation events; intermediate construction leaves it unchanged.
B.4 REWARD-JUMP MEMORY ARCHITECTURE
- The initial memory is computed once without gradient tracking, then remains unchanged between terminal events and jumps using the current memory and realized reward–outcome mark.Expression and pool encodings do not enter the jump directly, and the memory has no continuous drift or diffusion.Source: §B.4 · p. 13
B.5 SDE RETURN CRITIC
- The SDE critic receives an 88-dimensional state–action vector, simulates K Euler–Maruyama steps, and uses terminal particles to represent possible returns.The implementation uses independent Gaussian increments across particles.Source: §B.5 · pp. 13–14
B.6 EXPLORATION STRATEGY
- Exploration samples uniformly from available actions on the random branch; otherwise, the critic scores 62 selectable actions and masks unavailable ones.The exploitation branch selects the action with the highest particle score.Source: §B.6 · p. 14
B.7 CONTIGUOUS REPLAY AND OPTIMIZATION
- Replay stores pre-action memory and pool-snapshot identifiers; training reconstructs memory chronologically, using eight burn-in positions and eight learning positions.Jump regularization is aggregated over event-bearing positions, including burn-in, and is zero if no evaluation event occurs in the unroll.Source: §B.7 · pp. 14–15
B.9 ALPHARJM HYPERPARAMETER SETTINGS
- Universe-specific settings are reported in Table 3, while other parameters use common implementation settings; h = 42 runs reuse the corresponding h = 20 configuration.The same h = 42 reuse rule applies to the reported CSI800 and CSI1000 experiments.Source: §B.9 · p. 15, Table 3 · p. 15
C SENSITIVITY CONFIGURATION
This section describes a one-factor-at-a-time sensitivity protocol and outlines the benchmark and experiment details covered in the appendix.
C SENSITIVITY CONFIGURATION
- Sensitivity analysis on CSI300 with seed 0 varies one quantity at a time from the default configuration while holding all other parameters fixed.The supplied passage does not specify the varied quantities.Source: C SENSITIVITY CONFIGURATION · p. 15
- The reported sensitivity comparisons use the default operating point as their starting configuration.All other parameters remain fixed while one quantity is varied.Source: C SENSITIVITY CONFIGURATION · p. 15
D EXPERIMENT DETAILS
- Experiment details cover the common benchmark protocol, reported metrics, baseline implementations, and computational environment.AlphaRJM architecture and optimization settings are provided in Appendix B, with universe-specific settings in Table 3.Source: D EXPERIMENT DETAILS · p. 16
D.1 BENCHMARK PROTOCOL
The appendix specifies the shared benchmark protocol, evaluation measures, baseline implementations, and computing environment. Comparisons use common data and evaluation settings wherever applicable.
D.1 BENCHMARK PROTOCOL
- The main benchmark tests CSI300, CSI500, and CSI800 on a shared chronological split, with training through 2020, validation in 2021, and testing from 2022 to 2024.The standard forecasting horizon is h = 20, and the long-horizon study uses h = 42.Source: §D.1 · pp. 16–17
- AlphaQCM, AlphaSAGE, and AlphaRJM train for 10,000 terminal formula episodes, while AlphaGen retains PPO training with 50,000 requested primitive-action steps.AlphaGen's completed runs contain 51,200 executed primitive actions because PPO collects complete 2,048-step rollouts.Source: §D.1 · p. 17
D.2 EVALUATION METRICS
- IC and RIC are cross-sectional Pearson and rank correlations; ICIR and RICIR summarize their means relative to variation across test dates.Higher values indicate stronger predictive performance or temporal stability.Source: §D.2 · p. 17
- The portfolio visualization ranks stocks by predicted alpha each date and uses the top 20% of valid stocks; the same portfolio rule applies to all methods.When multiple seeds are shown, each seed's wealth trajectory is computed separately and the plotted curve is their pointwise mean.Source: §D.2 · p. 17
D.3 BASELINE IMPLEMENTATIONS
- Formula-discovery baselines retain their public reference implementations under the shared benchmark protocol, while Neural ODE and Neural SDE directly predict targets from market features.The Neural SDE baseline is distinct from AlphaRJM's critic, which models distributions of future symbolic-search returns.Source: §D.3 · p. 18
D.4 COMPUTATIONAL ENVIRONMENT
- Experiments ran locally on an Apple M4 Mac mini with 16 GB unified memory, using the reported Python, PyTorch, Qlib, and other package versions.The machine also had a 256 GB SSD.Source: §D.4 · p. 18
E.1 ABLATION STUDY
On CSI300 with seed 0, the full AlphaRJM model has the highest IC and ICIR among the ablation variants. Removing the critic, persistent memory, or distributional loss lowers performance.
E.1 ABLATION STUDY
- Replacing the SDE return critic with the QCM/IQN return critic produces the largest decline in both IC and ICIR.This variant retains event-conditioned memory.Source: §E.1 · p. 18, Table 4 · p. 19
- Removing persistent memory lowers both IC and ICIR, while removing the distributional-loss energy-distance term moderately reduces IC and more substantially reduces ICIR.The latter variant retains SDE dynamics and mean calibration.Source: §E.1 · p. 18, Table 4 · p. 19
- The full model achieves the highest IC and ICIR among the CSI300 seed-0 ablation variants.Table 4 reports the ablation study; higher values are better.Source: §E.1 · p. 18, Table 4 · p. 19
E.2 ADDITIONAL RESULTS ON CSI1000
The appendix reports CSI1000 correlation metrics, a CSI500 long-horizon wealth visualization, and theoretical properties of AlphaRJM’s memory and return critic.
E.2 ADDITIONAL RESULTS ON CSI1000
- 0.076 ± 0.003 IC and 0.102 ± 0.009 RIC are AlphaRJM’s highest CSI1000 scores; its RICIR is second-highest at 0.648 ± 0.013.AlphaQCM and LightGBM achieve the strongest ICIR and RICIR, respectively.Source: §E.2 · p. 19, Table 5 · p. 19
E.3 LONG-HORIZON CUMULATIVE WEALTH ON CSI500.
- Figure 4 presents cumulative wealth curves on CSI500 for forecasting horizon h = 42 using random seed 0.Source: Figure 4 · p. 20
F THEORETICAL PROPERTIES OF ALPHARJM
- The theoretical appendix establishes event-driven persistence and bounded variation for Reward-Jump Memory, characterizes the SDE critic, and analyzes Euler–Maruyama particle moments and statistic consistency.Source: F THEORETICAL PROPERTIES OF ALPHARJM · p. 20
F.1 EVENT-DRIVEN MEMORY DYNAMICS
The section formalizes event-driven memory persistence and bounds its variation, then establishes properties of the SDE critic and its finite-step particle estimates.
F.1 EVENT-DRIVEN MEMORY DYNAMICS
- Reward-Jump Memory stays constant between terminal evaluations, and its cumulative variation and net displacement are controlled by the number of evaluation events.Each update changes the memory by at most Jmax.Source: §F.1 · p. 20
F.2 SDE RETURN CRITIC
- For a fixed search state and available action, the SDE return critic has a unique strong solution whose conditional law is Gaussian with finite moments.Its mean moves from the initial value toward µt,a at a rate controlled by κt,a, while σt,a determines dispersion; this is an internal return, not an asset-price process.Source: §F.2 · pp. 22–23
F.3 EULER–MARUYAMA PARTICLES AND ACTION SCORING
- For fixed integration steps, independent Euler–Maruyama terminal particles are conditionally Gaussian, and their empirical mean and variance consistently estimate the finite-step return distribution’s moments.Monte Carlo estimation errors vanish almost surely as the number of particles grows, while the finite-step discretized process remains unchanged.Source: §F.3 · pp. 23, 24, 26