PaperlayerView exact revision on arXiv

Source-linked AI summary

PRICE: A Systematic Study of LLM Adaptation Choices for Bitcoin Price Forecasting

Maryam Fakhari, Mehran Safayani

arXiv:2609.05235v1cs.LGcs.AImath.NA

TL;DR

Volatile, non-stationary cryptocurrency markets and limited evidence on interacting LLM adaptation choices motivate PRICE for short-term Bitcoin forecasting. PRICE combines controlled evaluation of five adaptation components with a quantized LLaMA-3 8B backbone, achieving the lowest or tied-lowest reported validation errors and the lowest test errors among evaluated methods. The findings indicate that adaptation choices materially affect forecasting accuracy, reliability, and robustness.

  • Problem

    Cryptocurrency volatility and non-stationarity challenge forecasting, while prior work provides limited evidence on combined LLM adaptation choices in financial settings.

  • Method

    PRICE uses a 4-bit quantized LLaMA-3 8B model with LoRA, recursive inference, integer-rounded inputs, CTF prompting, and exact zero-temperature decoding.

  • Results

    PRICE achieves the lowest or tied-lowest reported validation errors and the lowest errors across all reported test metrics among evaluated forecasting methods.

  • Takeaways & Limitations

    Forecasting effectiveness depends on model adaptation, numerical representation, task formulation, and prediction generation, not only on the underlying LLM.

  • Takeaways & Limitations

    Results are limited to Bitcoin, hourly data, a fixed forecasting horizon, evaluated market periods, and the LLaMA-3 8B backbone.

Abstract

from arXiv · show

Cryptocurrency markets exhibit extreme volatility and non-stationary dynamics that challenge conventional forecasting methods. Although Large Language Models (LLMs) have shown promise for time series forecasting, the combined effects of adaptation choices remain largely unexplored in financial settings. This study introduces PRICE, a structured approach for adapting LLMs to short-term Bitcoin price forecasting. Built on a 4-bit quantized LLaMA-3 8B model, PRICE investigates how fine-tuning, numerical representation, prompting, inference, and decoding jointly influence forecasting performance. PRICE integrates Parameter-efficient fine-tuning with Low-Rank Adaptation (LoRA), Recursive multi-step inference, Integer-rounded numerical representation, Context-Task-Format (CTF) prompting, and Exact zero-temperature decoding. Ablation studies show that each component contributes to forecasting accuracy and reliability. LoRA enables efficient training on limited hardware, recursive inference improves accuracy, integer-rounded values reduce errors, CTF prompting outperforms Chain-of-Thought, Implicit Chain-of-Thought (iCoT), and few-shot prompting, and zero-temperature decoding improves stability during recursive forecasting. Comparative evaluation against eight transformer-based and time-series foundation models shows that PRICE achieves the lowest forecasting errors on both validation and test sets while maintaining robust performance across evaluation periods. Despite being based on a model primarily pretrained on text rather than time-series data, PRICE achieves competitive or superior performance relative to specialized foundation models. These findings demonstrate that adaptation choices critically determine the accuracy and robustness of LLMs for numerical time-series forecasting.

1 Introduction

Cryptocurrency forecasting is difficult because volatile, nonlinear, and non-stationary markets challenge conventional models. PRICE addresses limited evidence on interacting LLM adaptation choices through controlled ablations and achieves the lowest errors among evaluated methods.

  • Cryptocurrency markets exhibit high volatility, abrupt regime shifts, and sensitivity to speculative behavior and social sentiment.
  • Existing studies provide limited insight into how numerical representation, prompting, learning, inference, and decoding choices interact in financial LLM forecasting.
  • PRICE unifies five adaptation components around a 4-bit quantized LLaMA-3 8B model and evaluates their contributions under controlled ablations.
  • Fine-tuning improves predictive accuracy, while recursive multi-step inference improves forecasting performance and output reliability over direct multi-step generation.
  • Integer-rounded representations outperform raw, standard-normalized, and instance-normalized inputs by preserving price scale while reducing unnecessary precision.
  • CTF prompting outperforms CoT, iCoT, and few-shot prompting, while PRICE achieves the lowest forecasting errors on validation and test sets.

2 Related Works

Related work spans statistical, machine-learning, transformer, foundation-model, and LLM approaches to time-series forecasting. Prior LLM studies leave cryptocurrency-specific and systematic adaptation comparisons comparatively underexplored.

  • Traditional forecasting includes statistical models, machine-learning methods, deep neural networks, and transformer architectures with differing assumptions and temporal modeling capabilities.
  • Time-series foundation models use large-scale pretraining across heterogeneous datasets to support general-purpose or zero-shot forecasting.
  • LLM forecasting approaches include direct prompting, lightweight adaptation, parameter-efficient fine-tuning, and specialized numerical representations.
  • Prior studies primarily emphasize general benchmarks or traditional financial assets, leaving cryptocurrency forecasting and systematic evaluation of prompting, representation, and inference strategies comparatively underexplored.

3 Methodology

PRICE adapts a quantized LLaMA-3 8B model through one-step training and recursive multi-step inference, integer-rounded inputs, and structured CTF prompts. The pipeline is designed to align numerical forecasting inputs and outputs with language-model generation.

  • PRICE integrates LoRA fine-tuning, recursive inference, integer-rounded representation, CTF prompting, and exact zero-temperature decoding in one forecasting pipeline.
  • LoRA inserts trainable low-rank matrices into selected transformer layers while keeping most pretrained parameters frozen, reducing computational and memory requirements.
  • Recursive Multi-Step Inference: Training uses a one-step-ahead objective, whereas inference recursively appends each prediction and removes the oldest observation until the forecasting horizon is reached.
  • Sliding windows transform ordered time-series subsequences into historical inputs and subsequent forecasting targets, advancing with stride one.
  • Integer-Rounded Numerical Representation: Integer-rounded representation rounds each Bitcoin price to the nearest integer before converting the sequence into textual model input.
  • Context-Task-Format Prompting: CTF prompting structures instructions around context, task, and format, specifying Bitcoin hourly closing prices, the next-price objective, and a single numerical output.
  • Context-Task-Format Prompting: Candidate prompt variants are evaluated on validation data, after which the strongest configuration is fixed for subsequent experiments.
  • Zero-temperature decoding removes stochastic variation and supports stable outputs during recursive forecasting.

4 Experimental Setup

The experiment evaluates short-term Bitcoin closing-price forecasting on a chronologically partitioned hourly dataset using a quantized, LoRA-adapted LLaMA-3 model and complementary output-processing and scaling procedures.

  • Dataset and preprocessing: 2018–2024 Binance minute-level OHLCV records were aggregated into hourly closing prices, with the final experimental period restricted to March 2023–December 2024.The restriction aimed to provide a more consistent market environment and reduce distribution shifts from long-term cryptocurrency-market changes.
  • Dataset and preprocessing: The final dataset was divided chronologically into training, validation, and test sets using an 80%/10%/10% ratio.Chronological splitting preserves temporal ordering and prevents information leakage.
  • Model and implementation: The forecasting model is a 4-bit quantized LLaMA-3 8B-Instruct model adapted with LoRA while keeping pretrained parameters frozen.Quantization supports memory-efficient fine-tuning on limited hardware, while low-rank adapters are inserted into selected linear layers.
  • Dataset and preprocessing: Each sample uses 96 historical hourly closing prices to predict the subsequent 8 observations, forming a four-day input and an 8-hour forecasting horizon.Samples were generated with a sliding window of stride 1.
  • Evaluation: Forecast quality was assessed with complementary metrics, including validity filtering and standardized scale-dependent comparisons across models.Invalid outputs were removed when they contained non-numeric text, malformed values, or an incorrect prediction length; predictions and targets were standardized using training-set statistics for scale-dependent metrics.
  • Evaluation: SMAPE was interpreted on a [0, 200] scale because its denominator averages the absolute prediction and ground-truth values.Values above 100 indicate predictions with opposite sign to the true values.

5 Experiments and Results

The experiments show that numerical representation, forecasting strategy, prompt design, and decoding-related choices materially affect PRICE's forecasting accuracy and output reliability. Fine-tuning, recursive multi-step inference, rounded inputs, and structured prompting provide the strongest configurations across the evaluated settings.

  • Effect of Input Representation: Rounded values achieve the lowest errors across all reported representation metrics, with MAPE 3.01, SMAPE 3.05, and NRMSE 0.05.They outperform raw, standard-normalized, and instance-normalized inputs while preserving the original price scale with reduced numerical precision.
  • Effect of Learning Paradigm: Fine-tuning reduces MAE from 2063.62 to 534.85 for single-step forecasting and from 1779.88 to 445.60 for multi-step forecasting.These reductions correspond to approximately 74% and 75%, respectively, under the two forecasting strategies.
  • Effect of Forecasting Strategy: Multi-step autoregressive forecasting achieves lower MAE than single-step forecasting across learning paradigms and substantially reduces MSE after fine-tuning.Both multi-step configurations achieve VNP=1441, whereas zero-shot single-step forecasting yields 1182 valid predictions.
  • Effect of Prompt Design: iCoT keeps structured analytical guidance in the prompt while constraining output to a single numerical prediction, avoiding externally visible reasoning traces.It remains compatible with LoRA fine-tuning and autoregressive forecasting workflows, but the fine-tuned CTF configuration performs better.
  • Effect of Prompt Design: The CTF prompt achieves the lowest fine-tuned MAE of 432.83 and MSE of 402,655.83, outperforming iCoT and the basic prompt while preserving VNP=1441.The few-shot configuration has the largest errors among zero-shot methods, while CoT produces the weakest LLaMA zero-shot performance and lowest VNP of 1,403.

6 Conclusion

PRICE shows that coordinated adaptation choices materially affect LLM forecasting accuracy, reliability, and robustness for short-term Bitcoin prediction. Across ablations and model comparisons, it performs strongly while remaining limited to the evaluated asset, horizon, periods, and LLaMA-3 8B backbone.

  • PRICE combines LoRA fine-tuning, recursive multi-step inference, integer-rounded values, CTF prompting, and exact zero-temperature decoding in one forecasting framework.
  • Each adaptation dimension materially influences numerical forecasting accuracy or reliability, with LoRA, recursive inference, integer rounding, CTF prompting, and deterministic decoding improving their evaluated outcomes.The reported effects include improved accuracy, output validity, representation performance, prompting performance, and recursive-forecast stability.
  • PRICE achieves the lowest or tied-lowest errors across reported validation metrics and the lowest errors across all test metrics against transformer and time-series foundation models.
  • Systematic evaluation indicates that adaptation, numerical representation, task formulation, and prediction generation are important determinants of forecasting accuracy, output reliability, and robustness.
  • The framework supports general-purpose LLMs as competitive forecasters even when they are not specifically designed or pretrained for time-series prediction.
  • The findings are bounded by experiments on one asset, hourly data, one forecasting horizon, selected market periods, and one LLaMA-3 8B backbone.The authors call for evaluation across additional assets, resolutions, horizons, backbones, and financial or non-financial series.
Loading 2609.05235v1…