PaperlayerView exact revision on arXiv

Source-linked AI summary

Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach

Haowei Ni, Shuchen Meng, Xupeng Chen, Ziqing Zhao, Andi Chen, Panfeng Li, Shiyao Zhang, Qifu Yin, Yuanqing Wang, Yuxi Chan

arXiv:2408.06634v2q-fin.CPcs.AIcs.CLcs.LGq-fin.ST

TL;DR

The paper addresses the difficulty of predicting stock movements after earnings announcements from complex, lengthy reports and other market information. It fine-tunes an LLM using earnings-related inputs and external factors; the llama-3-8b-Instruct-4bit model is reported to perform best on the Full dataset among the evaluated models.

  • Problem

    Traditional machine-learning models struggle to interpret extensive, unstructured earnings-report text and capture its implications for stock prediction.

    Source: §I · p. 1

  • Method

    The approach instruction-fine-tunes an LLM on earnings reports and financial metrics together with external factors such as market performance and analyst grades, using QLoRA compression.

    Source: §I · p. 2, A. Framework · p. 3, §V · p. 5

  • Results

    The llama-3-8b-Instruct-4bit model demonstrated superior performance in the Full dataset among the evaluated models.

    Source: 2) Comparative Analysis of Model… · p. 5

  • Takeaways & Limitations

    The study concludes that integrating AI techniques with a curated financial dataset offers improvements over traditional forecasting models.

    Source: §V · p. 5

    The model’s binary Long/Short output may not capture nuanced investor behavior, and its next-day horizon does not cover longer-term movements.

    Source: §V · p. 5

Abstract

from arXiv · show

Accurate stock market predictions following earnings reports are crucial for investors. Traditional methods, particularly classical machine learning models, struggle with these predictions because they cannot effectively process and interpret extensive textual data contained in earnings reports and often overlook nuances that influence market movements. This paper introduces an advanced approach by employing Large Language Models (LLMs) instruction fine-tuned with a novel combination of instruction-based techniques and quantized low-rank adaptation (QLoRA) compression. Our methodology integrates 'base factors', such as financial metric growth and earnings transcripts, with 'external factors', including recent market indices performances and analyst grades, to create a rich, supervised dataset. This comprehensive dataset enables our models to achieve superior predictive performance in terms of accuracy, weighted F1, and Matthews correlation coefficient (MCC), especially evident in the comparison with benchmarks such as GPT-4. We specifically highlight the efficacy of the llama-3-8b-Instruct-4bit model, which showcases significant improvements over baseline models. The paper also discusses the potential of expanding the output capabilities to include a 'Hold' option and extending the prediction horizon, aiming to accommodate various investment styles and time frames. This study not only demonstrates the power of integrating cutting-edge AI with fine-tuned financial data but also paves the way for future research in enhancing AI-driven financial analysis tools.

I. INTRODUCTION

Historical-only forecasting and conventional machine-learning models struggle with dynamic post-earnings markets and lengthy reports; the paper proposes fine-tuning an LLM on earnings text and external market signals.

  • Historical data alone and traditional indicators such as moving averages can be inadequate, particularly when significant events disrupt markets.Source: §I · p. 1
  • Traditional methods struggle to capture complex market conditions and the implications of extensive, unstructured earnings-report text.Investors also face the volume of information and rapid market reactions around earnings announcements.Source: §I · pp. 1–2
  • Prior financial NLP work includes RAG-based sentiment analysis [4], SEP explainable stock prediction, and FinBERT-LSTM news-sentiment forecasting [6].Source: §I · p. 2
  • The paper proposes fine-tuning an LLM on earnings reports and external factors to produce actionable insight into post-earnings stock movements.Source: §I · p. 2

II. DATA COLLECTION AND PREPROCESSING

The study assembled data for 501 S&P 500-listed companies, combining market, analyst, earnings, financial-statement, and transcript information with a next-day stock-direction label.

  • The dataset covers 501 S&P 500-listed companies, with financial data collected through the Financial Modeling Prep API.Source: A. Dataset Curation · p. 2
  • For each earnings date, the collection includes prior-week performance for SPY, QQQ, DOW, and the company’s stock, plus analyst upgrades and downgrades from preceding months.Source: A. Dataset Curation · p. 2
  • Earnings surprises are identified by comparing actual earnings per share with estimates.Source: A. Dataset Curation · p. 2
  • Financial inputs include year-over-year quarterly growth across income statements, balance sheets, and cash-flow statements, alongside full earnings-call transcripts.Source: A. Dataset Curation · p. 2
  • The next-day label is Long when the opening price is below the closing price and Short otherwise.Source: A. Dataset Curation · p. 2

B. Textualization and Tokenization

The preprocessing phase converts numerical and categorical financial data into descriptive text for LLM input, within a framework that also includes fine-tuning, QLoRA compression, and evaluation.

  • Numerical and categorical data are textualized as descriptive sentences to make financial information more contextual and easier for the model to process.Source: B. Textualization and Tokenization · p. 2
  • The framework overview includes dataset setup and instruction-based fine-tuning before QLoRA compression.Source: Figure 1 · p. 3
  • The overview also includes evaluation using designated prompts and outputs.Source: Figure 1 · p. 3

1) Market Performance Metrics:

Market-performance inputs are textualized as descriptive statements, while analyst grades are aggregated over the preceding 30 days and expressed in text.

1) Market Performance Metrics:

  • Percentage changes for major indices and individual stocks were converted into descriptive statements, and this transformation was applied consistently to SPY, QQQ, DOW, and the specific company ticker’s performance data.Original data: “SPY -1.5%”; rendered as “In the past week, SPY went down by 1.5%”Source: 1) Market Performance Metrics · p. 3

2) Analyst Grades:

  • Analyst grades: the most frequent recommendation over the past 30 days is aggregated and rendered as a descriptive statement.The example describes the most frequent grade as a recommendation to buy the stock.Source: 2) Analyst Grades · p. 3

3) Earnings Surprises:

Earnings surprises and year-over-year financial growth were converted into descriptive text, and the resulting data was organized into Base and Full datasets to compare internal and external inputs.

3) Earnings Surprises:

  • Actual and estimated EPS values were turned into comparative statements; one example reports earnings per share 7.69% above analysts’ consensus estimates.Source: 3) Earnings Surprises · p. 3

4) Financial Metrics Growth:

  • Year-over-year growth ratios were converted into descriptive sentences, including that Net Income grew 4.6% versus the same quarter a year earlier.Source: 4) Financial Metrics Growth · p. 3
  • After rows with missing values were removed, the dataset contained 8,556 rows, with an average of 12,375 tokens per row.Source: 4) Financial Metrics Growth · p. 3
  • The Base dataset uses financial-metric growth and earnings transcripts, while the Full dataset adds market performance, analyst grades, and earnings estimates.The split was designed to assess the relative impact of internal versus external data on predictive accuracy.Source: 4) Financial Metrics Growth · p. 3

III. METHOD

The method combines company fundamentals and external financial signals in a supervised dataset, then instruction-fine-tunes compressed pretrained models and evaluates binary predictions against market data.

A. Framework

  • The supervised dataset combines financial-metric growth and earnings transcripts with prior-week market and stock performance, analyst grades, and earnings surprises.Source: A. Framework · p. 3
  • Models compressed with QLoRA were fine-tuned using targeted instructions, and structured prompts produced binary outputs during inference.Source: A. Framework · p. 3
  • Evaluation against actual market data used accuracy, weighted F1 score, and Matthews correlation coefficient (MCC); Table I is titled as a comparison of different models.Source: A. Framework · p. 3, Table I · p. 4

B. Instruction Fine Tuning

Instruction fine-tuning trains models to follow structured financial-prediction instructions, while 4-bit QLoRA reduces adaptation cost for deployment under resource constraints.

B. Instruction Fine Tuning

C. QLoRA

  • The study uses 4-bit QLoRA to compress adaptive parameters while aiming to preserve model performance and support resource-constrained deployment.Source: C. QLoRA · p. 4
  • QLoRA reduces the precision of low-rank matrices from 32 bits to 4 bits, decreasing the model’s memory footprint.Source: C. QLoRA · p. 4
  • Lower-bit operations are described as less computationally expensive, supporting faster computation and more efficient operation in resource-limited settings.Source: C. QLoRA · p. 4

IV. EVALUATION

The evaluation setup fine-tuned several pretrained models with a shared training protocol and included GPT-4.0 as a benchmark.

A. Model Training

  • Six pretrained models, including llama-3-8b-Instruct-4bit, were fine-tuned for one epoch at a learning rate of 2e −4.Source: A. Model Training · p. 4
  • Training used gradient accumulation, a five-step warm-up, AdamW 8-bit, weight decay of 0.01, and a linear learning-rate schedule.Source: A. Model Training · p. 4

1) ChatGPT:

  • GPT-4.0 was accessed through OpenAI’s API as a performance benchmark.Source: 1) ChatGPT · p. 4

C. Performance Analysis

The llama-3-8b-Instruct-4bit model outperformed GPT-4 on the Full dataset, while fine-tuning loss consistently declined. The section evaluates models using accuracy, weighted F1, and MCC.

1) Evaluation Metrics:

  • The evaluation uses accuracy, weighted F1, and Matthews Correlation Coefficient (MCC) to assess predictive performance.Weighted F1 accounts for precision and recall and is useful when class imbalance may affect evaluation; MCC captures true and false positives and negatives.Source: 1) Evaluation Metrics · p. 5

2) Comparative Analysis of Model Performance:

  • On the Full dataset, llama-3-8b-Instruct-4bit achieved 16% higher accuracy and 10% better Weighted F1 than GPT-4.The Full dataset includes external factors alongside base factors.Source: 2) Comparative Analysis of Model… · p. 5
  • Fine-tuning loss consistently decreased across training steps, which the authors describe as indicating effective learning and adaptation.Figure 2 shows the training loss curve for the Llama 3-8b-Instruct model.Source: 2) Comparative Analysis of Model… · p. 5, Figure 2 · p. 5

V. CONCLUSION AND FUTURE WORK

The study concludes that instruction fine-tuning with QLoRA and combined base and external factors improves post-earnings stock forecasts, with the llama-3-8b-Instruct-4bit model outperforming GPT-4 on several metrics. It identifies binary Long/Short outputs and next-day prediction as scope boundaries for future work.

V. CONCLUSION AND FUTURE WORK

  • The llama-3-8b-Instruct-4bit model outstripped GPT-4 in accuracy, weighted F1, and MCC after combining instruction fine-tuning, QLoRA, and base and external factors.The Full dataset includes broader contextual information than the Base dataset.Source: §V · p. 5
  • QLoRA compression is presented as a way to deploy the models on resource-constrained devices while maintaining high predictive accuracy and minimizing operational demands.Source: §V · p. 5
  • The binary Long/Short output may not capture real-world investor behavior fully, and future work proposes adding Hold and extending forecasts to weekly movements.Source: §V · p. 5
  • The paper characterizes its integrated AI and curated dataset approach as improving substantially over traditional models.Source: §V · p. 5
  • The authors identify integrating more diverse data types and advanced modeling techniques as directions for further financial-domain research.Source: §V · p. 6
Loading 2408.06634v2…