PaperlayerView exact revision on arXiv

Source-linked AI summary

Regime-Aware Portfolio Management via Retrieval-Augmented LLM-Guided Expert Switching

Ahmad Asadi, Reza Safabakhsh

arXiv:2608.28252v1cs.AI

TL;DR

Non-stationary markets make expert effectiveness depend on changing conditions, motivating adaptive portfolio routing. The paper retrieves similar historical states and their expert outcomes, then uses an LLM to select among specialized experts; across cryptocurrency, stocks, and foreign exchange, this selector achieves the strongest cumulative return and Sharpe ratio among evaluated strategies.

  • Problem

    Existing routing signals may rely on current representations, learned gates, or noisy recent performance, while LLM-based financial agents remain inconsistent as direct trading controllers.

  • Method

    The framework retrieves similar market states, evaluates expert performance in those episodes, and uses a dual-stream Variational Autoencoder plus LLM reasoning to select an expert without generating portfolio weights.

  • Results

    The selector achieves the highest cumulative return and Sharpe ratio across cryptocurrency, stocks, and foreign exchange; a four-expert pool improves Sharpe by 19.8% in cryptocurrency, 22.0% in stocks, and 27.5% in foreign exchange.

  • Takeaways & Limitations

    Historical retrieval-grounded switching supports adaptive portfolio management, and complementary expertise improves performance as the expert pool expands.

  • Takeaways & Limitations

    Each market experiment is a single run, so reported values are point estimates rather than averages over multiple random seeds.

Abstract

from arXiv · show

Financial markets are inherently non-stationary, making the effectiveness of individual portfolio-management strategies highly dependent on changing market conditions. This work proposes a retrieval-augmented expert-switching framework that dynamically selects portfolio management experts based on their historical performance under similar market situations. A dual-stream variational autoencoder represents asset-level and market-wide information, while a retrieval-based knowledge base stores historical situations and expert performance. During inference, an instruction-tuned LLM reasons over the retrieved evidence to identify the most appropriate expert rather than directly generating portfolio actions. We further establish a monotonicity property showing that adding a locally superior expert cannot degrade the switching mechanism's performance. Experiments across cryptocurrency, stock, and foreign-exchange markets show that the proposed selector achieves the highest cumulative return and Sharpe ratio among the evaluated selection strategies in all three markets. In the stock market, for example, cumulative return increases from 26% for the best fixed expert to 34%, while the Sharpe ratio improves from 0.74 to 0.96. Ablation results confirm the importance of both retrieval and LLM reasoning, while experiments with different expert-pool sizes demonstrate the value of complementary expertise. Overall, the findings support retrieval-grounded expert switching as an effective approach to adaptive portfolio management in non-stationary financial environments.

1 Introduction

Non-stationary markets challenge fixed and monolithic portfolio strategies because training-time regularities may fail under shocks and unseen regimes. The proposed framework instead retrieves comparable historical situations and switches among specialized experts using risk-aware evidence.

  • Non-stationary financial data, abrupt shocks, irrational behavior, and cross-asset dependencies complicate portfolio management.
  • Deep reinforcement learning and mixture-of-experts methods can adapt decisions, but learned systems may overfit training-specific regularities or misroute experts in unseen regimes.
  • The framework treats expert selection as a sequential decision problem grounded in historical situations and risk-adjusted expert performance.
  • A dual-stream Variational Autoencoder represents asset-level and market-wide information for retrieval-based switching, while an LLM selects an expert rather than directly generating portfolio actions.
  • Adding an expert that outperforms the existing pool within a sub-domain guarantees non-degradation of overall performance and supports incremental expert-pool expansion.

2 Literature Review

Prior work uses learned policies, regime models, expert gates, recent performance, and LLM coordination to address changing markets. The paper identifies unresolved reliability and routing issues and motivates historical retrieval with a bounded LLM decision-support role.

  • Mean-variance methods rely on approximately Gaussian returns and stable correlations, assumptions challenged by volatility clustering, structural breaks, and changing dependencies.
  • Deep reinforcement learning removes separate prediction and optimization stages, while later architectures model temporal structure, behavioral assumptions, or overestimation bias.
  • Regime-aware and mixture-of-experts systems maintain heterogeneous policies, but switching quality is critical because slow or fast regime responses can misselect experts.
  • Recent-performance routing is inexpensive and objective-related but noisy, allowing transient returns to favor experts poorly matched to emerging conditions.
  • LLM financial systems support information processing and coordination, whereas large-scale benchmarks do not show consistent outperformance of buy-and-hold strategies.
  • The proposed research direction retrieves similar historical states, evaluates expert behavior there, and assigns the LLM a bounded contextual selection role.

3 Proposed Method

The proposed system separates historical indexing from online expert selection. It stores latent market states with forward expert performance, then uses current-state encoding, nearest-neighbor retrieval, and expert scoring for routing.

  • Historical market windows are encoded into latent state representations, evaluated across experts over a fixed forward horizon, and stored as state–performance pairs in a vector database.
  • The offline index is constructed once per historical window, while online inference performs state encoding, nearest-neighbor retrieval, and expert scoring.
  • Retrieved observations provide an explicit record linking expert selection to behavior under historically similar market conditions.

3.2 System Architecture and Data Flow

The architecture has separate offline and online pipelines: historical states and expert outcomes form the retrieval index, while current-state processing retrieves comparable episodes and routes to an expert.

  • Figure 1 separates historical-index construction in the lower pipeline from current-state processing and expert selection in the upper pipeline.
  • Asset-level streams include returns, momentum, volatility, and volume, while market-wide streams encode aggregate trend, volatility, and liquidity.
  • Separate encoders combine asset-level and market-wide representations into a single vector h_t while preserving both individual-asset and market-wide information.
  • Historical records pair each representation with every expert’s realized fixed-horizon performance and a Statement of Performance for online routing.
  • At inference, approximate nearest-neighbor search retrieves comparable states, and the LLM assesses the records without generating portfolio weights.

3.3 Theoretical Framework: Expert Switching with Historical Risk Evaluation

The framework routes each market state to an expert using retrieved historical episodes, risk-adjusted performance assessments, and conservative switching rules. Under stated assumptions, adding a locally superior specialist cannot reduce expected utility and can improve performance in its target region.

  • Historical market states are represented over a latent-state distribution, while each expert maps observed market representations to portfolio weights and has historically estimated risk-adjusted utility.
  • Historical Evidence and Switching Rule: The router retrieves similar historical episodes, combines observed expert outcomes with contextual LLM assessment, and uses the resulting scores to select or retain the active expert.The LLM assessment calibrates retrieved numerical evidence rather than directly observing unobservable true utility.
  • Conditions for Non-Decreasing Performance: The switching mechanism assumes bounded estimation error, conservative replacement only beyond a threshold, and a specialist with a sufficiently large advantage in a positive-probability target region.The specialist condition requires δ > τ + 2ϵ, while the switching threshold satisfies τ ≥ 2ϵ.
  • Operational Interpretation of the Conditions: Prompt constraints operationalize the assumptions by grounding assessments in observed returns and drawdowns, applying switching margins, and checking specialist performance during tail events and outside its intended regime.The paper distinguishes these prompt instructions from proofs of the mathematical assumptions and describes them as implementation heuristics.
  • Monotonicity Result: V(π_N+1) ≥ V(π_N): under Axioms 1–3, adding a specialist does not decrease expected utility and yields strict improvement on its target region.Repeated additions produce a non-decreasing sequence bounded above by optimal achievable utility; with coverage of the relevant state space, the limit approaches V*.
  • Monotonicity Result: If specialist target regions cover the relevant state space while satisfying the dominance conditions, successive expert additions converge toward the optimal achievable utility.

3.4 Prompt Design and Uncertainty Calibration

The prompt design constrains LLM risk assessments to retrieved empirical evidence and makes switching decisions conservative and tail-aware. Validation reduced mean absolute estimation error after these empirical-range constraints were introduced.

  • Empirical Grounding: The final prompt prevents extrapolated returns and uses historical baseline means when retrieved evidence for a regime is sparse.
  • Uncertainty Calibration: Approximately 18% to 11%: mean absolute estimation error decreased across five prompt iterations after empirical-range constraints were introduced.
  • Empirical Grounding: Risk and return assessments are grounded in historical maximum drawdown, realized Information Ratios, and other performance observations supplied in retrieved context.
  • Conservative Margining: A newly introduced expert receives a risk penalty and is preferred only when its estimated advantage exceeds the specified noise margin.
  • Tail-Event Analysis: The evaluator separately examines extreme drawdowns, liquidity squeezes, and regime transitions, requiring specialist advantages to remain localized without severe deterioration elsewhere.
  • Structured Output: The LLM outputs structured JSON so its assessment can enter numerical routing without free-form text parsing.

3.5 Market State Decomposition and Feature Modeling

The framework decomposes market state into separate asset-level and market-wide feature streams over rolling windows. Keeping these streams distinct preserves complementary information for identifying comparable historical states.

  • Feature streams: Market state uses two rolling-window tensors: one for asset-specific technical behavior and one for market-wide conditions.The asset-level tensor spans L × A × F1, while the market-wide tensor spans L × F2.
  • Asset-level features: Asset-level features include log-returns, RSI, MACD, ATR, realized volatility, and volume changes.
  • Market-wide features: Market-wide features contain aggregate trend indicators, volatility-regime measures, and liquidity variables.
  • Feature separation: Separating the streams preserves relative instrument behavior and shared portfolio conditions for historical state matching.The representation avoids forcing both information types into a single input representation.
  • Normalization: Rolling z-score normalization limits scale-change effects and reduces heteroskedasticity's influence on state representation.

3.6 Dual-Stream Transformer-VAE Embedding

Each feature stream is encoded with a pretrained Transformer-VAE, and the resulting latent vectors are combined into a shared market-state representation. The same representation function is used for historical indexing and current-state retrieval.

  • Encoding: Each asset-level and market-wide stream is encoded with a pretrained Transformer-VAE.The encoder maps each stream to a Gaussian posterior whose parameters are generated by a temporal Transformer encoder.
  • Pretraining: During pretraining, sampled latent representations are decoded to reconstruct inputs; afterward, the decoder is removed.The retained encoder is used for market-state representation.
  • Latent fusion: The two stream-specific latent vectors are concatenated into a joint representation.
  • Retrieval space: Using the same representation function for historical and current states places retrieval distances in a common latent space.Historical states use hτ, while the current state uses ht before nearest-neighbor retrieval.

3.7 Offline Indexing

Offline indexing converts historical market situations into searchable state embeddings paired with forward expert-performance records and Statements of Performance.

  • Historical evaluation: For each historical time step, the system constructs a market-state representation and evaluates every expert over a forward horizon H.
  • Performance records: Each indexed record stores cumulative return, Sharpe ratio, and maximum drawdown for the evaluated expert episodes.
  • Performance statements: A Statement of Performance records market context, observed expert behavior, and relevant uncertainty factors from the same episode.
  • Index structure: An index entry pairs a historical latent state with expert-performance information.

3.8 Online Retrieval and Portfolio Decision

Online decision-making represents the current market window, retrieves similar historical states, and supplies their performance evidence to an LLM. The LLM selects an existing expert according to the current situation and retrieved risk-return profiles.

  • Online retrieval: The current market window is encoded as ht, after which the K closest historical states are retrieved.
  • Offline preparation: The offline procedure constructs dual-stream embeddings, executes each expert over horizon H, evaluates realized performance, and stores the resulting index records.
  • Performance estimation: Retrieved episodes receive normalized similarity weights for estimating each expert's performance.The estimates are based on similarity between the current state ht and retrieved states hτ.
  • Evidence-grounded selection: The LLM receives retrieved Statements of Performance, performance summaries, and the current market description.
  • Portfolio decision: The model selects the best existing expert based on the current market situation and experts' retrieved risk-return behavior profiles.

4 Experimental Results

Experiments evaluate the retrieval-augmented expert-switching framework across cryptocurrency, stock, and foreign-exchange markets, including comparisons, ablations, and sensitivity analyses. The proposed selector achieves the strongest reported risk-return performance across the evaluated markets, while retrieval, LLM reasoning, and complementary expert pools materially affect results.

  • Experimental Setup: The framework is evaluated independently on cryptocurrency, stock, and foreign-exchange markets using 30-symbol panels and comparisons with baseline portfolio-management methods.Each market experiment uses the complete 30-symbol panel and is conducted once, so reported values are point estimates from single runs.
  • Similarity-Driven Expert Selection: The proposed selector improves cumulative return and Sharpe relative to the best fixed expert across all three markets.Cumulative return rises from 68% to 71% and Sharpe from 1.64 to 1.73 in cryptocurrency; from 26% to 34% and 0.74 to 0.96 in stocks; and from 3.2% to 4.3% and 0.71 to 0.88 in foreign exchange.
  • Ablation: Number of Expert Models: Four-expert pools produce the largest reported Sharpe improvements in every market: 19.8% in cryptocurrency, 22.0% in stocks, and 27.5% in foreign exchange.The results indicate that complementary expertise, rather than model count alone, is the relevant pool-design variable.
  • Component Ablation: Ablations show that retrieval and LLM reasoning both contribute to performance, with their relative importance differing across markets.Removing retrieval reduces cryptocurrency cumulative return from 71% to 64.8%, while removing the LLM reduces stock return to 25.5% and Sharpe from 0.96 to 0.735; in foreign exchange, removing the LLM produces −0.2% return and −0.025 Sharpe.
  • LLM Sensitivity: Model Used for Inference: Inference-model quality materially affects expert selection, with Qwen3.6-27B-it outperforming the evaluated Gemma4 variants across all three markets.In foreign exchange, cumulative return changes from −5.4% and −0.3% with the two Gemma variants to 4.3% with Qwen3.6-27B-it, while Sharpe rises from −1.07 and −0.05 to 0.88.
  • Comparison with State-of-the-Art Methods: The proposed framework achieves the strongest overall risk-adjusted performance among compared methods in cryptocurrency, stocks, and foreign exchange.The stock comparison reports 34% cumulative return, Sharpe 0.96, annualized volatility 0.19, and MDD −26%; the foreign-exchange comparison reports Sharpe 0.88, volatility 0.03, and MDD −2.2%.

5 Conclusion

The experiments support retrieval-grounded expert switching for adaptive portfolio management across cryptocurrency, stock, and foreign-exchange markets. They also indicate contributions from complementary experts, retrieval, and LLM reasoning, while leaving uncertainty-aware decision making unisolated.

  • The proposed RAG-based selector achieves the highest cumulative return and Sharpe ratio across cryptocurrency, stocks, and foreign exchange.No single expert consistently dominates across all markets and evaluation metrics.
  • Performance improves as complementary experts are introduced, highlighting the importance of diversity within the candidate pool.
  • Ablation results indicate that both retrieval and LLM reasoning contribute substantially to the framework’s effectiveness.
  • The experiments do not isolate the independent contribution of uncertainty-aware decision making.The reported ablation omits a configuration removing only the uncertainty component.
  • Future evaluation should cover multiple random seeds, market periods, transaction-cost scenarios, and statistically tested repeated experiments.
Loading 2608.28252v1…