Source-linked AI summary
Loss Choice or Model Choice? The Role of Forecast Level in Cryptocurrency Volatility Forecasting
Andrzej Tokajuk, Jarosław A. Chudziak
TL;DR
The paper addresses how training-loss choice compares with model choice when raw volatility scores can mix forecast-level shifts and daily forecast movements. It compares seven losses and five models, using validation-based alignment to assess raw and aligned forecasts. Much of the raw loss effect reflects forecast level, while after alignment model differences are larger in the full grid and cross-loss VaR breach-rate differences narrow.
Problem
Because loss choice can change forecast level as well as daily movements, raw comparisons leave unclear how much differences reflect each component.
Source: §1 · pp. 1–2Method
The study compares seven losses and five models across cryptocurrencies, aligns forecast levels using validation data, and evaluates raw and aligned forecasts.
Source: §3.2 · p. 4Results
After alignment, marginal score differences are larger across model configurations in the full grid, while cross-loss differences in one-day VaR breach rates narrow substantially.
Source: §7 · p. 7Takeaways & Limitations
Reporting raw and aligned results distinguishes differences associated with forecast level from those remaining after adjustment and clarifies their connection to risk estimates.
Source: §7 · pp. 7–8Evidence covers five cryptocurrencies, one-day forecasts, five models, and seven losses; constant alignment cannot correct time-varying bias, and VaR depends on OHLC-based proxies and a Gaussian return model.
Source: §6 · p. 7
Abstract
from arXiv · showhide
Volatility forecasts play a central role in financial risk management because their overall level and day-to-day movements affect downstream decisions. Most studies compare forecasting models while keeping the training loss fixed. Yet losses emphasise different errors and can target different properties of future volatility, so raw comparisons may combine persistent forecast-level differences with differences in daily forecast movements. This leaves unresolved whether the importance of loss choice comes mainly from the forecast level it targets or from differences that remain after level adjustment. We address this gap through a comparison of seven losses and five models across major cryptocurrencies. Validation-based alignment adjusts the forecast level before the raw and aligned forecasts are evaluated using statistical scores and one-day Value-at-Risk. Before alignment, marginal score variation is greater across losses. After alignment, model choice becomes the larger source of variation in the full five-model comparison, while cross-loss differences in VaR breach rates narrow substantially. Our contribution is a comprehensive evaluation of loss and model choice that shows why losses can appear so influential in raw comparisons and how this interpretation changes when forecast level and downstream risk are considered explicitly.
1 Introduction
The paper asks whether training-loss choice matters beyond the forecast level it targets, since raw score differences can mix level shifts with daily forecast movements. It compares losses and models, aligns forecast levels, and evaluates statistical scores and one-day VaR.
1 Introduction
- Volatility forecasts inform position sizing, capital allocation, and VaR, and similar daily paths can imply substantially different levels of financial risk.Their usefulness depends on both overall level and day-to-day movements.Source: §1 · p. 1
- Raw score differences may combine persistent forecast-level shifts with differences in daily forecast movements, complicating statistical evaluation and practical risk use.Forecast level feeds directly into risk measures.Source: §1 · pp. 1–2
- The study compares seven losses and five models across five major cryptocurrencies, aligning forecast levels with validation data before evaluating scores and one-day VaR.Non-loss settings are held fixed in the walk-forward comparison.Source: §1 · p. 2
2 Related Work
Prior volatility-forecasting research has emphasized model design, while giving less attention to training-loss choice. Existing work leaves open how loss choice compares with model choice and how much of the difference reflects forecast level.
2 Related Work
- Prior model comparisons report mixed findings, with results varying by market, predictors, asset, evaluation metric, and forecast horizon.HAR remains a strong baseline across horizons.Source: §2 · p. 2
- Different losses can target different conditional quantities, and volatility rankings may also depend on the evaluation score and proxy used.Volatility is not observed directly and must be approximated using a proxy.Source: §2 · p. 2
- Prior work leaves open how training-loss choice compares with model choice and how much of the difference reflects forecast level in volatility forecasting.The paper examines both questions and their implications for VaR.Source: §2 · p. 2
3 Methodology
The methodology compares losses and models, then uses forecast-level alignment to distinguish persistent level differences from daily dynamics. It also characterizes the forecasts targeted by each loss.
3.1 Loss Functions
- Each loss can favor a different one-day-ahead forecast target, and the paper describes targets and error properties for its loss functions.MSE-log targets the conditional geometric mean; Huber-log gives less weight to large log errors, while HMAE weights observations by h.Source: §3.1 · p. 3, Table 1 · p. 3
3.2 Comparison and Forecast-Level Alignment
- The primary five-model comparison uses native predictors, so model choice includes differences in inputs and history length; shared inputs serve as a robustness check.All models are compared under all seven losses with fixed settings.Source: §3.2 · p. 3
- Marginal loss and model score differences are averaged across all 21 loss pairs and 10 model pairs, with their median ratio over 25 blocks summarizing relative variation.Ratios above one indicate greater differences across losses; this descriptive comparison does not separate loss-model interactions.Source: §3.2 · pp. 3–4
- A validation constant rescales each loss-model forecast to change its level while preserving relative daily movements.The constant minimizes validation QLIKE over constant rescalings; LevelShare separately distinguishes mainly level differences from time-varying differences.Source: §3.2 · p. 4
4 Experimental Setup
The experiments use daily cryptocurrency OHLCV data, five forecasting models, and a walk-forward design with fixed non-loss settings. QLIKE is the primary score, with alternative scores, proxies, and one-day Gaussian VaR also evaluated.
4 Experimental Setup
- Daily Binance USDT spot OHLCV data cover BTC, ETH, BNB, XRP, and ADA through 31 May 2026, with next-day unannualised Garman-Klass variance as the target.Parkinson and Rogers-Satchell proxies test measurement robustness.Source: §4 · p. 4
- The five models are HAR, HAR-X, NeuralHAR, DLinear, and LightGBM, with differing predictors and target-history inputs.HAR uses daily, weekly, and monthly log variance; DLinear uses 48-day target history.Source: §4 · p. 4
- Five expanding folds provide 1,013–2,529 training days, 253 validation days, and 379 test days, with disjoint test windows.Validation controls early stopping and alignment, and all 25 asset-fold blocks receive equal weight.Source: §4 · p. 4
- QLIKE is the primary score, with MSE-log, HMAE, alternative proxies, and raw and aligned one-day 5% Gaussian VaR also evaluated.The Kupiec test assesses asset-level VaR coverage.Source: §4 · p. 5
5 Results
Raw comparisons showed larger marginal score differences across losses, but forecast-level alignment made model choice the larger source of variation in the full grid. The results also show that alignment narrows cross-loss differences in one-day VaR breach rates, though correct coverage is not established for every series.
5.1 RQ1: Loss Choice Is a Major Source of Raw Variation
- The median loss-to-model variation ratio was 2.91 (95% CI [2.27, 3.25]); it remained above one with shared predictors at 2.73 (95% CI [2.01, 3.44]).Across assets, the ratio ranged from 1.44 to 4.55.Source: §5.1 · p. 5
- QLIKE and Patton generally led under QLIKE evaluation, while HMSE performed poorly for HAR and HAR-X but remained competitive for LightGBM.The loss-to-model ratio remained above one under MSE-log (3.71) and HMAE (3.98); loss rankings were similar across the reported volatility proxies.Source: Table 2 · p. 5, §5.1 · p. 5
5.2 RQ2: Forecast Level Explains Most Raw Loss Variation
- Alignment reduced cross-loss variation by 59%-88% across models and 77% overall, shifting the loss-to-model ratio from 2.91 to 0.67.With shared predictors, the estimate was 0.81 (CI [0.66, 1.22]), so the reversal was statistically clear only in the full grid.Source: Table 3 · p. 6, §5.2 · p. 6
5.3 RQ3: Forecast-Level Differences Propagate to Value-at-Risk
- Loss rankings under QLIKE and MSE-log had Spearman correlation −0.200; raw VaR breach rates spanned 1.3%-6.3%, narrowing to 3.5%-3.6% after alignment.Alignment removed 97% of cross-loss breach-rate variation, but the Kupiec test still rejected correct coverage in 109 of 175 aligned asset-loss-model series.Source: Table 4 · p. 6, §5.3 · pp. 6–7
6 Discussion
Cross-loss dynamics depend on the model: forecast level explains most HAR differences, while LightGBM shows more loss-specific daily variation; the evidence is bounded by its markets, horizon, and risk-model assumptions.
6 Discussion
- Forecast level explains most HAR differences, whereas LightGBM shows more loss-specific daily variation.The balance between level and dynamics depends on the model.Source: §6 · p. 7
- Raw results capture the complete loss effect; aligned results show what remains after validation-based level adjustment, including possible residual test-period level differences.Source: §6 · p. 7
- The evidence covers five cryptocurrencies, one-day forecasts, five models, and seven losses; fixed settings, constant alignment, and OHLC-based Gaussian VaR constrain interpretation.The balance may differ across markets or horizons, and constant alignment cannot correct time-varying bias.Source: §6 · p. 7
7 Conclusion and Future Work
The study finds that forecast level accounts for much of the raw loss effect, with model-dependent dynamics; after alignment, model configurations show larger marginal score differences in the full grid, while VaR breach-rate differences narrow.
7 Conclusion and Future Work
- After alignment, marginal score differences are larger across model configurations in the full grid, while the shared-input comparison remains inconclusive.Alignment also substantially narrows cross-loss differences in one-day VaR breach rates.Source: §7 · p. 7
- Reporting raw and aligned results explains why a loss can appear superior under a statistical score without necessarily learning better temporal dynamics.The joint analysis considers loss choice, model choice, and forecast level.Source: §7 · p. 7
- Future work can extend the analysis to other markets and horizons, loss-specific tuning, and non-Gaussian risk models.Source: §7 · p. 8