PaperlayerView exact revision on arXiv

Source-linked AI summary

Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting

Xinli Yu, Zheng Chen, Yuan Ling, Shujing Dong, Zongyi Liu, Yanbin Lu

arXiv:2306.11025v1cs.LGcs.AIcs.CLq-fin.ST

TL;DR

Financial forecasting still faces challenges in cross-sequence reasoning, multimodal signal integration, and interpretable results. This paper tests GPT-4 inference and fine-tuned Open LLaMA with structured financial inputs; its experiments report that LLMs outperform conventional baselines, with chain-of-thought reasoning improving performance.

  • Problem

    Financial forecasting methods face challenges in capturing cross-sequence dependencies, integrating multimodal financial signals, and providing human-readable explanations.

    Source: §1 · pp. 1–2

  • Method

    The paper uses structured prompts combining stock prices, company information, and financial news with GPT-4 zero-/few-shot inference and instruction-fine-tuned Open LLaMA.

    Source: §3 · p. 3, §5 · p. 8

  • Results

    LLMs surpassed ARMA-GARCH and gradient-boosting models, chain-of-thought reasoning enhanced performance, and GPT-4 few-shot with COT performed best in accuracy and explanation quality.

    Source: §4.2 · p. 5, §5 · p. 8

  • Takeaways & Limitations

    Fine-tuning can make a publicly available LLM generate explainable forecasts with reasonable performance on this financial forecasting task.

    Source: §4.2 · p. 5, §5 · p. 8

    Fine-tuned Open LLaMA is competitive with GPT-4 in binary precision but has worse bin precision and higher MSE from more extreme predictions.

    Source: §4.2.1 · p. 8

Abstract

from arXiv · show

This paper presents a novel study on harnessing Large Language Models' (LLMs) outstanding knowledge and reasoning abilities for explainable financial time series forecasting. The application of machine learning models to financial time series comes with several challenges, including the difficulty in cross-sequence reasoning and inference, the hurdle of incorporating multi-modal signals from historical news, financial knowledge graphs, etc., and the issue of interpreting and explaining the model results. In this paper, we focus on NASDAQ-100 stocks, making use of publicly accessible historical stock price data, company metadata, and historical economic/financial news. We conduct experiments to illustrate the potential of LLMs in offering a unified solution to the aforementioned challenges. Our experiments include trying zero-shot/few-shot inference with GPT-4 and instruction-based fine-tuning with a public LLM model Open LLaMA. We demonstrate our approach outperforms a few baselines, including the widely applied classic ARMA-GARCH model and a gradient-boosting tree model. Through the performance comparison results and a few examples, we find LLMs can make a well-thought decision by reasoning over information from both textual news and price time series and extracting insights, leveraging cross-sequence information, and utilizing the inherent knowledge embedded within the LLM. Additionally, we show that a publicly available LLM such as Open-LLaMA, after fine-tuning, can comprehend the instruction to generate explainable forecasts and achieve reasonable performance, albeit relatively inferior in comparison to GPT-4.

1 INTRODUCTION

Financial forecasting poses challenges in cross-sequence reasoning, multimodal data integration, and interpretability. The paper explores LLMs for explainable NASDAQ-100 forecasting, testing GPT-4 inference and Open LLaMA fine-tuning against existing methods.

1 INTRODUCTION

  • Existing approaches face challenges capturing intricate dependencies across time series, while varying designs and data requirements can hinder their wider application.Prior approaches include correlation analysis, clustering, and deep learning.Source: §1 · p. 1
  • Financial forecasting also involves integrating diverse signals and producing human-readable explanations, which existing deep-learning methods often do not provide.The cited examples of signals include historical news, financial knowledge graphs, and social media; attention-based explanations still require substantial interpretation.Source: §1 · pp. 1–2
  • The paper motivates LLMs as potential tools for reasoning across sequences and handling varied financial data, while generating explanations through natural language and step-by-step reasoning.The authors describe these abilities as potential ways to address the challenges, rather than established solutions.Source: §1 · p. 2
  • The study explores LLMs for explainable financial time series forecasting on NASDAQ-100 stock prices, using GPT-4 inference and instruction-based Open LLaMA fine-tuning.The authors also report that chain-of-thoughts improves performance in most experiments.Source: §1 · p. 2
  • GPT-4 zero-shot inference outperforms a gradient-boosting tree model using about 300 features.The paper compares its LLM approaches with ARMA-GARCH and gradient-boosting baselines.Source: §1 · p. 2

2 RELATED WORKS

Related work spans statistical and econometric forecasting, machine-learning methods, and emerging LLM applications. The paper positions explainable financial forecasting with LLMs as relatively unexplored.

2 RELATED WORKS

  • The paper situates LLM forecasting within a field shaped by statistical, machine-learning, and deep-learning approaches.It describes financial time series forecasting as an extensively researched area with multiple methodologies.Source: §2 · p. 2

2.1 Traditional Statistical/Econometric Methods

  • Traditional statistical and econometric forecasting includes widely used ARMA-GARCH models and alternatives such as VAR, state-space models, and VECM.ARMA-GARCH is described as capturing dependencies and volatility clustering in financial time series.Source: §2.1 · p. 2

2.2 Machine Learning Techniques

  • Machine-learning work ranges from decision trees and support-vector machines to RNNs, CNNs, and Transformers applied to financial forecasting.Recent deep-learning methods are described as capturing complex, nonlinear data relationships.Source: §2.2 · p. 2

2.3 Large Language Models

  • LLMs have demonstrated reasoning over heterogeneous data and generation of human-readable explanations, but explainable financial forecasting remains relatively unexplored.The paper presents its work as a contribution to this emerging area.Source: §2.3 · p. 2

3 METHODOLOGY

The methodology combines NASDAQ-100 price histories with company information and financial news to forecast weekly or monthly returns with explanations. It uses structured prompting for GPT-4 inference and instruction fine-tuning for Open LLaMA.

3 METHODOLOGY

  • The study forecasts weekly or monthly NASDAQ-100 stock returns with accompanying explanations, using company metadata and stock-specific and broader financial news.Returns are defined as percentage changes from the beginning to the end of the week or month.Source: §3 · pp. 2–3

3.1.1 Stock Price Data. We download daily NASDAQ-100 stock1

  • Price changes are normalized as percentage changes and categorized into directional bins, with bin granularity varying by forecast horizon.The weekly scheme has 12 bins, while the monthly scheme can extend to D10+ and U10+ categories.Source: §3.1.1 · p. 3

3.1.2 Company Profile Data.

  • Company profiles include GPT-4-generated descriptions and general positive and negative factors that may affect stock prices.Weekly stock news is summarized and converted into keywords by GPT-4.Source: §3.1.2 · p. 3, §3.1.3 · p. 3

3.1.3 Finance/Economy News Data.

  • Weekly macroeconomic and financial news is summarized into a condensed meta-summary and keywords for the experiments.The weekly meta-summary is generated from the top-story summaries and keywords.Source: §3.1.3 · p. 3

3.2 Instruction-Based Zero-shot/Few-shot Inference with LLMs

  • GPT-4 zero-shot and few-shot prompts combine instructions, temporal news, categorized price series, and examples from similar stocks to support forecasting.The predicted summary and keywords serve as explanations, and the authors report that step-by-step reasoning improved performance by a few points.Source: §3.2 · p. 3, §3.1.1 · p. 3

3.3 Instruction-based Fine-tuning with Open LLaMA

  • Open LLaMA 13B is instruction-fine-tuned because its zero-shot responses tend to repeat the prompt instead of following its instructions.Training examples use historical weekly and monthly forecasts; prompts are compressed to 1K tokens with GPT-4-generated summaries, and Open LLaMA's extracted summaries and keywords often fall short of usability.Source: §3.3 · pp. 3–4
  • After fine-tuning, Open LLaMA produces more coherent forecasts and explanations and is competitive with GPT-4 on binary classification, but its extreme predictions yield relatively higher squared error.The examples illustrate generated company profiles, news summaries, and weekly meta-summaries.Source: Figure 1 · p. 4, Figure 2 · p. 4, §3.3 · p. 5

4 EXPERIMENTS

The experiments evaluate LLM-based stock-return forecasting and explanations against baseline models, using a five-year training window and a 52-week evaluation window. GPT-4 few-shot with chain-of-thought performs best overall, while fine-tuned Open LLaMA is competitive on some measures but weaker on exact-bin precision and error magnitude.

4.1.1 Data Time Window.

  • The evaluation uses 52 weeks from June 6, 2022 to June 4, 2023, after training and fine-tuning on five years of data from June 12, 2017 to June 5, 2022.The training window is also used to train the baseline models.Source: §4.1.1 · p. 5

4.1.2 Baseline Models.

  • The forecasting baselines include a most-frequent historical-bin heuristic, ARMA-GARCH (p=1, q=1), and a LightGBM gradient-boosting tree using about 300 features.The tree features include daily price and volume data, rolling-window statistics, sector information, and historical earnings.Source: §4.1.2 · p. 5

4.1.3 Evaluation Metrics.

  • Forecasts are evaluated weekly and monthly, with four weeks treated as a month; metrics include binary precision, exact-bin precision, and mean squared error of consecutive bin ordinals.ROUGE-1 and ROUGE-2 compare generated next-period news summaries and keywords with GPT-4-extracted summaries and keywords from actual top news.Source: §4.1.3 · p. 5

4.2 Performance Evaluation

  • GPT-4 few-shot with COT consistently performs best in prediction accuracy and explanation quality, while GPT-4 and fine-tuned Open LLaMA outperform ARMA-GARCH and gradient boosting on binary and bin precision.The forecasting comparison is in Table 1; Table 2 evaluates explanation quality.Source: §4.2 · p. 5, §4.2.1 · p. 5

4.2.1 Stock Price Forecasting.

  • GPT-4 few-shot with COT achieves the best forecasting performance and the lowest MSE, while fine-tuned Open LLaMA is competitive in binary precision but weaker in bin precision and has higher MSE from more extreme predictions.The authors report that GPT-4 few-shot with COT best predicts direction and more accurately estimates the magnitude of price changes.Source: §4.2.1 · p. 8

4.2.2 Explanation Qality.

  • GPT-4 few-shot with COT achieves the highest ROUGE scores; fine-tuned Open LLaMA with COT also shows reasonable explanation quality compared with GPT-4 without COT.Table 2 evaluates ROUGE-1 and ROUGE-2 for generated news summaries and keywords.Source: §4.2.2 · p. 8

5 CONCLUSION

The conclusion presents LLM-based NASDAQ-100 return forecasting with human-readable explanations, using structured prompts that combine company profiles, stock-price histories, and financial news. The authors report performance gains over traditional and machine-learning baselines and describe the results as preliminary.

  • The approach uses structured prompts combining company profiles, historical stock prices, and financial news to guide GPT-4 and Open LLaMA in forecasting NASDAQ-100 stock returns and generating explanations.Source: §5 · p. 8
  • The authors report that LLMs surpass ARMA-GARCH and gradient-boosting baselines, and that step-by-step reasoning enhances LLM performance.Their Open LLaMA fine-tuning experiments demonstrate feasibility for publicly available models on this task.Source: §5 · p. 8
  • The authors characterize the results as preliminary and propose testing other indexes, adding data types such as macroeconomic series and social-network data, and fine-tuning larger public models.Source: §5 · p. 8

ARMA-GARCH

The section labels forecasting-model variants and identifies tables for weekly/monthly stock-price forecasting and explanation-quality evaluation.

ARMA-GARCH

  • The listed model labels include a gradient-boosting tree and fine-tuned Open LLaMA 13B, both with and without COT.Source: ARMA-GARCH · p. 9
  • Table 1 is a performance comparison of baseline models and LLMs for weekly and monthly stock-price forecasting.Source: Table 1 · p. 9
  • Table 2 evaluates explanation quality using ROUGE scores against GPT-4 summaries and keywords extracted from each week’s actual top news.Source: Table 2 · p. 9

Related papers

Loading 2306.11025v1…