Source-linked AI summary
Retrieval-Augmented Diffusion Modeling for Stochastic Discount Factor Portfolios
Kelvin J. L. Koa, Xinyang Li, Ke-Wei Huang
TL;DR
SDF portfolio optimization needs market representations that capture risk structure despite regime shifts, noisy price and news inputs, and state-dependent financial uncertainty. RADAR retrieves similar historical regimes to construct context-dependent diffusion noise and denoise multimodal representations. The paper reports state-of-the-art risk-adjusted performance and economically meaningful signals on returns and correlations.
Problem
SDF portfolio optimization requires representations of shared market-state and risk structure, while financial regimes shift, price and news inputs are noisy, and standard isotropic diffusion noise misses state-dependent uncertainty.
Source: §1 · pp. 1–2, §3.3 · p. 5Method
RADAR retrieves similar historical regimes to construct context-dependent noise, then conditionally denoises price and news representations using retrieved-context statistics for initialization.
Source: §1 · p. 2Results
RADAR consistently achieves state-of-the-art Sharpe, Sortino, and Calmar ratios and produces economically meaningful signals on asset returns and correlations.
Source: §1 · p. 2Takeaways & Limitations
The experiments support RADAR's use for SDF-based portfolio optimization with representations that reflect current-market risk–return characteristics.
Source: §1 · p. 2Experiments rely on the relatively sparse FNSPID news dataset, and limited news coverage can affect the quality of learned representations.
Source: §6 · p. 10
Abstract
from arXiv · showhide
In this work, we study portfolio optimization under the stochastic discount factor (SDF) framework by learning market state representations that capture the underlying risk structures of financial data. This is challenging due to several factors: financial markets exhibit non-stationary dynamics with shifting regimes, multimodal inputs such as price and news data often contain stochastic noise, and existing diffusion-based approaches, while effective for modeling stochastic dynamics, rely on assumptions such as isotropic Gaussian noise that fail to capture the state-dependent nature of financial uncertainty. To address these challenges, we introduce RADAR, a retrieval-augmented diffusion framework that learns market representations by conditioning on similar historical regimes. RADAR leverages retrieval to construct context-dependent noise distributions, applies conditional diffusion to denoise multimodal representations, and initializes the diffusion process using empirical statistics to reflect state-dependent uncertainty. Experiments show that RADAR achieves state-of-the-art performance on key risk-adjusted metrics while producing economically meaningful signals on asset returns and correlations.
1 Introduction
The introduction frames SDF portfolio optimization as learning market representations that capture shared risk structure despite regime shifts, noisy multimodal inputs, and limitations of standard diffusion noise. RADAR retrieves similar historical regimes to guide denoising and reports strong risk-adjusted performance with economically meaningful signals.
1 Introduction
- SDF estimation is underdetermined and requires representations that capture joint market-state and risk structure to satisfy pricing constraints across assets, unlike asset-specific returns forecasting.The supplied introduction contrasts forecasting conditional expectations with SDF estimation's pricing constraints.Source: §1 · pp. 1–2
- Market regime shifts can bias models toward better-covered periods, while noisy textual inputs can hinder economically meaningful signals and suboptimal pricing and allocation.Existing temporal weighting and trend-decomposition methods are described as heuristic and unable to capture contextual similarities across market states.Source: §1 · p. 2
- Existing diffusion methods typically use isotropic Gaussian noise that misses state-dependent, heteroskedastic financial uncertainty, while RATD conditions denoising on retrieved states rather than using them to guide noise generation.The cited comparison also notes that RATD does not handle multimodal inputs.Source: §1 · p. 2
- RADAR conditions diffusion on similar historical regimes, denoises price and news representations, and initializes the process with retrieved-context means and variances.The framework aims to reflect the current market state's risk–return characteristics in representations for downstream SDF estimation.Source: §1 · p. 2
- RADAR consistently achieves state-of-the-art performance on Sharpe, Sortino, and Calmar ratios, with learned signals on asset returns and correlations that benefit portfolio outcomes.The introduction also reports economically meaningful representations for portfolio construction.Source: §1 · p. 2
2 Related Works
Related work positions RADAR at the intersection of financial forecasting, portfolio optimization, and diffusion modeling. It distinguishes the framework by integrating information-driven signals with stochastic-noise modeling and by constructing noise distributions from retrieved historical samples.
Financial Forecasting
- Financial forecasting combines textual signals and diffusion-noise modeling in separate lines of work, whereas RADAR integrates both for portfolio applications.The passage situates text as a way to capture market trends and volatility, and diffusion noise as a model of stochastic price noise.Source: §2 · p. 3
Deep Learning for Portfolio Optimization
- Portfolio methods range from mean–variance optimization, HRP, and reinforcement learning to SDF pricing models that use shared asset structure to capture risk premia and state-dependent trade-offs.The passage identifies non-stationarity, noisy inputs, and context-dependent financial risk as practical challenges for asset representations.Source: §2 · p. 3
Non-Isotropic Diffusion Processes
- Prior non-isotropic diffusion work modifies noise schedules, forward processes, or data transformations, while RADAR constructs its noise distribution directly from retrieved historical samples.Source: §2 · p. 3
3 The RADAR Framework
RADAR learns market representations for SDF-based portfolio optimization from price and news inputs. It retrieves similar historical market contexts to guide diffusion-based denoising and downstream portfolio learning.
3.1 Problem Formulation
- The framework maps price and news inputs into joint market-state embeddings used to learn asset portfolio weights under an SDF objective.The objective minimizes the second moment of the SDF, which the paper states makes it price every asset with zero error.Source: §3.1 · p. 3
3.2 Multimodal Encoders
- GRU encoders with attention produce price and text representations, which a bilinear transformation blends into a joint representation.The text encoder uses embeddings from pre-trained financial language models.Source: §3.2 · p. 4
3.3 Contextual Noise Retrieval
- RADAR organizes news events and their subsequent price sequences into historical segments, encodes them, and stores their representations in a context bank.Each segment ends before the next news event arrives.Source: §3.3 · p. 5
3.4 Retrieval-Augmented Diffusion
- Retrieval supplies context-specific noise statistics for dual-path diffusion, replacing input-independent isotropic noise with perturbations based on similar historical market states.The model retrieves similar segments across modalities and uses the retrieved sets’ means and variances as noise parameters.Source: §3.4 · p. 5, §3.3 · p. 5
- Denoisers reconstruct price and text representations while conditioning on their shared market representation, enabling cross-modal information exchange.The denoised modality-specific representations are aggregated into a refined market representation.Source: §3.4 · pp. 5–6
3.5 Overall Training Objective
- Training jointly optimizes the diffusion reconstruction objective and the SDF-based portfolio objective, with ρ balancing representation learning and portfolio optimization.The diffusion objective reconstructs modality-specific representations from noisy counterparts for subsequent market representation construction.Source: §3.5 · p. 6, §3.4 · p. 6
4 Experiments
The experiments evaluate RADAR on a rolling out-of-sample portfolio task using price and financial-news data. Comparisons span forecasting and end-to-end portfolio models, with risk-adjusted metrics as the main indicators of portfolio quality.
Dataset
- The evaluation uses 377 U.S. equities with daily adjusted prices and financial news from June 2011 to June 2020, in rolling four-year training and one-year test windows.Each training window uses a 90:10 train-validation split and rolls forward by one year to avoid look-ahead bias.Source: §4 · p. 6
Baselines
- Forecasting baselines include StockNet, HAN, NGAT, iTransformer, and RATD, whose return forecasts form portfolios equally weighted across the top 50 stocks.RATD is the closest related work; unlike RADAR, it uses retrieved sequences as conditional guidance rather than to construct context-dependent diffusion noise.Source: §4 · p. 6
- End-to-end portfolio baselines include BSV, DKKM, MLP, LinAttn, and SDF, which learn portfolio weights directly from input features.SDF performed best in the original SDF work, as reported in the passage.Source: §4 · p. 6
Evaluation and Implementation
- Portfolio quality is assessed mainly with Sharpe, Sortino, and Calmar ratios, alongside returns, maximum drawdown, and volatility.Implementation uses 60 trading days of input, a seven-day rebalancing horizon, and the top 10 retrieved neighbors.Source: §4 · pp. 6–7
5 Results
RADAR attains the best performance on most risk-adjusted metrics, while analyses indicate that retrieval, multimodal inputs, and denoised representations contribute to portfolio performance. Its gains persist across tested input lengths and rebalancing horizons, with results varying by domain.
Performance Comparison
- Forecasting models have close performance, while portfolio methods show Sharpe improvements with greater expressiveness and cross-asset information sharing.Forecasting models can realize higher volatility; portfolio methods show lower drawdowns and volatility, but lower Sharpe ratios than deep-learning models.Source: §5 · p. 7
- RADAR attains the best performance on most metrics, including Sharpe, Sortino, and Calmar ratios, and lower drawdowns than forecasting models.Table 1 compares performance; the authors characterize RADAR as having the best overall risk-return trade-off.Source: §5 · p. 7, Table 1 · p. 7
Ablation Study
- Removing either price or news inputs degrades performance; embeddings improve most metrics over the SDF baseline, while unrelated Gaussian diffusion noise reduces performance.The No Price variant has relatively low volatility; the authors suggest noisy price data contributes to volatility. Table 2 presents the ablation study.Source: §5 · p. 8, Table 2 · p. 8
Parameter Selection
- Small K yields unstable performance across seeds, whereas large K reduces mean Sharpe; K = 10 provides strong performance with low variability.The authors attribute small-K variability to sensitivity to a few retrieved neighbors and large-K deterioration to less relevant samples diluting the signal.Source: §5 · pp. 8–9, Figure 4 · p. 8
Market Representation Study
- RADAR embeddings show a clearer, more monotonic increase in realized return correlation across similarity deciles than noisy embeddings.Asset pairs are ranked by embedding cosine similarity and grouped into deciles; reported values aggregate five rolling windows.Source: §5 · p. 9, Figure 5 · p. 9
- Q1, the portfolio of highest-scoring assets, consistently outperforms lower-scoring quintiles in cumulative NAV.At each rebalancing date, assets are ranked by embedding-derived scores, divided into equal-weighted quintile portfolios, and tracked using forward returns.Source: §5 · p. 9, Figure 6 · p. 9
Robustness to Context and Target Lengths
- RADAR has the highest Sharpe ratio and cumulative return across all six tested input-length and rebalancing-horizon configurations.Performance generally decreases as the rebalancing horizon grows, but RADAR remains above forecasting baselines.Source: §5 · p. 9, Figure 7 · p. 10
Generalizability to Other Domains
- RADAR is largely generalizable across domains, although No Diffusion performs best in some Security and Traffic cases.The authors suggest those domains may have less noise or noise closer to Gaussian, and identify this as a topic for further study.Source: §5 · p. 10, Table 3 · p. 10
6 Conclusion
The paper presents RADAR for SDF portfolio optimization, using retrieved historical regimes to condition denoising of multimodal market representations. It reports strong risk-adjusted performance and economically meaningful representation signals, while noting limits tied to sparse news coverage and interpretability.
Conclusion
- RADAR learns context-aware market representations for SDF portfolio optimization by conditioning multimodal denoising on similar historical regimes.The paper motivates this approach by shifting market regimes, noisy price and news inputs, and isotropic noise that does not reflect state-dependent financial uncertainty.Source: §6 · p. 10
- RADAR achieves strong performance on key risk-adjusted metrics, with representations that align with asset co-movement and returns structure.Source: §6 · p. 10
Limitations
- Experiments rely on the relatively sparse FNSPID news dataset, and embedding-based portfolio weights may reduce interpretability and feature-importance insight.The paper notes that more comprehensive proprietary news sources could reduce replicability and suggests post-hoc explanation techniques as future work.Source: §6 · p. 10
A Diffusion Process Formulation
The forward process uses retrieved statistics to parameterize Gaussian corruption, while the reverse process estimates clean representations through denoising. Its simplified x0-prediction objective drops timestep- and covariance-dependent weighting.
A Diffusion Process Formulation
- Retrieved statistics parameterize Gaussian noise in place of the standard N(0, I), while the shifted, rescaled process remains standard diffusion in new coordinates.The text identifies the retrieved statistics as (µNq,t, σNq,t).Source: A Diffusion Process Formulation · p. 14
- The forward process defines marginal and stepwise corruption using context-dependent mean and noise scale, then expresses it as standard diffusion after centering and rescaling.Source: A Diffusion Process Formulation · p. 14
- During reverse sampling, the denoiser substitutes its estimate of the clean representation for the unavailable true representation and samples a reverse step.The same formulation applies to the text path.Source: A Diffusion Process Formulation · p. 14
- The denoising network predicts the clean representation, and Equation (14) simplifies its objective by dropping timestep- and covariance-dependent weighting.This follows established diffusion practices, including unweighted squared-error objectives [18], [19].Source: A Diffusion Process Formulation · p. 15
B Statistical Significance and Robustness
RADAR’s improvements over baselines are reported as statistically significant across pooled tests and robustness checks. The evaluation uses tests suited to return time series, Sharpe ratios, temporal dependence, and variation across random seeds.
B Statistical Significance and Robustness
- Monthly t-tests aggregate daily returns before testing, while Newey–West HAC tests adjust for autocorrelation and heteroskedasticity in return differences.The monthly test reduces microstructure noise and short-term dependence; HAC adjustment addresses violations of the i.i.d. assumption.Source: B Statistical Significance and… · p. 15
- Diebold–Mariano tests compare loss sequences over time, and Ledoit–Wolf tests compare Sharpe ratios while accounting for mean and covariance estimation error.Source: B Statistical Significance and… · p. 15
- Block bootstrap resampling preserves temporal dependence, while Fisher aggregation checks whether significance holds across random seeds.The bootstrap Sharpe test assesses Sharpe-ratio improvements under resampling.Source: B Statistical Significance and… · p. 15
- RADAR’s improvements are statistically significant across baselines and testing procedures, including monthly aggregation, HAC-adjusted inference, Sharpe-based evaluation, block bootstrap, and cross-seed aggregation.Bootstrap Sharpe and Ledoit–Wolf tests also support statistical significance of the Sharpe-ratio improvements.Source: B Statistical Significance and… · p. 16, Table 4 · p. 16, Table 5 · p. 16
C Cross-Sectional Performance Analysis
RADAR performs strongly across volatility groups and sectors, with the reported advantages especially pronounced in high-volatility settings and sectors with complex interactions. Its gains are more limited in Energy and Real Estate, where external macro factors may be less captured by firm-level representations.
C Cross-Sectional Performance Analysis
- RADAR achieves the highest annualized returns across volatility groups and the highest Sharpe ratios in low- and high-volatility groups, remaining competitive in the middle group.The annualized-return gap widens as volatility increases; the Sharpe trend does not strictly follow returns.Source: C Cross-Sectional Performance Analysis · p. 16, Figure 8 · p. 17
- Retrieval conditions on historically similar regimes, while diffusion denoising suppresses idiosyncratic noise; the text links these mechanisms to stronger stability and performance in high volatility.Source: C Cross-Sectional Performance Analysis · p. 17
- RADAR performs strongest in most sectors, with particularly large gains in Information Technology and Financials, but more limited gains in Energy and Real Estate.The authors suggest macroeconomic data may improve the latter sectors’ coverage; RADAR remains competitive in stable sectors such as Utilities and Consumer Staples.Source: C Cross-Sectional Performance Analysis · p. 17
- Overall, the analysis associates RADAR’s gains with modeling state-dependent noisy environments, especially high-volatility assets and sectors with complex interactions.The text also attributes grounding of representations to initialization with empirical conditional statistics.Source: C Cross-Sectional Performance Analysis · p. 18
D Financial Analysis of Portfolios
Financial-factor analyses indicate that RADAR’s returns are not explained solely by market exposure or common size and value factors. The reported alpha, beta, and R2 patterns are presented as evidence of economically meaningful signals beyond those factor exposures.
D Financial Analysis of Portfolios
- The analysis uses CAPM and Fama–French factor regressions to decompose returns into systematic risk-factor components and residual alpha.The paper treats positive, statistically significant alpha as excess returns beyond common factors and lower R2 as less return variation explained by those factors.Source: D Financial Analysis of Portfolios · pp. 18–19
- 18.71% annualized alpha for RADAR is statistically significant at the 5% level, while its beta is 0.977 and its R2 is 0.476.The text contrasts RADAR’s R2 with Equal Weight’s values above 0.95, indicating a larger unexplained share of return variation for RADAR.Source: D Financial Analysis of Portfolios · pp. 18–19, Table 6 · p. 19
- Under the Fama–French specification, RADAR has statistically significant alpha of 15.47%, relatively small SMB exposure, negative HML loading, and the lowest average R2 among baselines.The authors interpret this as performance not attributable to common size and value factors.Source: D Financial Analysis of Portfolios · p. 19, Table 7 · p. 19