PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Survey of Regression Based Loss Functions for Time Series Forecasting

Aryan Jadon, Avinash Patil, Shruti Jadon

arXiv:2211.02989v1cs.LGcs.AI

TL;DR

Time series forecasting requires suitable objectives for varied data conditions, but the appropriate loss function remains uncertain. The paper surveys 14 regression losses, organizes their useful circumstances, and evaluates them across four datasets. Its results show that the best-performing losses vary by dataset, while quantile loss performs well in each reported dataset.

  • Problem

    Time series forecasting involves varied conditions such as biased data, outliers, skewness, and unknown distributions, making loss-function selection uncertain.

    Source: §I · p. 1, §V · p. 11

  • Method

    The paper summarizes 14 regression loss functions, identifies circumstances for their use, and compares them across four datasets with Temporal Fusion Transformers.

    Source: §I · p. 1, §V · p. 12

  • Results

    Quantile loss performed well on the Electricity, traffic, Favorita, and volatility datasets, while other top-performing losses differed by dataset.

    Source: B. Performance Metrics · p. 11

  • Takeaways & Limitations

    Loss functions should be selected according to dataset properties and objectives rather than treated as universally interchangeable.

    Source: §V · pp. 11–12

Abstract

from arXiv · show

Time Series Forecasting has been an active area of research due to its many applications ranging from network usage prediction, resource allocation, anomaly detection, and predictive maintenance. Numerous publications published in the last five years have proposed diverse sets of objective loss functions to address cases such as biased data, long-term forecasting, multicollinear features, etc. In this paper, we have summarized 14 well-known regression loss functions commonly used for time series forecasting and listed out the circumstances where their application can aid in faster and better model convergence. We have also demonstrated how certain categories of loss functions perform well across all data sets and can be considered as a baseline objective function in circumstances where the distribution of the data is unknown. Our code is available at GitHub: https://github.com/aryan-jadon/Regression-Loss-Functions-in-Time-Series-Forecasting-Tensorflow.

I. INTRODUCTION

The paper surveys regression loss functions for time series forecasting, where temporal structure and data characteristics complicate objective selection. It summarizes 14 losses and compares them across forecasting datasets using Temporal Fusion Transformers.

  • Time series forecasting uses historical time-stamped data to learn patterns that inform future decisions.Source: §I · p. 1
  • The paper summarizes 14 regression loss functions and describes circumstances where each may support forecasting optimization.Source: §I · p. 1, §III · p. 2
  • The study compares all loss functions on Electricity, traffic, grocery-sales, and volatility datasets using Temporal Fusion Transformers.Source: §I · p. 1
  • Loss-function choice remains uncertain because forecasting data and objectives vary in distribution, boundaries, outliers, and computational requirements.Source: §I · p. 1
  • Forecasting models must learn level, cyclicity, trend, and seasonality while avoiding random noise.Source: §I · p. 1, §II · p. 2

B. Mean Squared Error (MSE)

MSE squares prediction errors, giving larger mistakes greater weight and producing a nonnegative objective that optimization algorithms can use. The supplied passages also contrast MSE with bias error, whose signed errors may cancel.

D. Relative Absolute Error (RAE)

Relative Absolute Error (RAE) normalizes total absolute prediction error by the absolute deviation of actual values from their mean. It ranges from 0 to 1, with values near zero indicating better models.

F. Mean Absolute Percentage Error (MAPE)

Mean Absolute Percentage Error (MAPE) expresses forecast error as an average absolute percentage, preventing positive and negative errors from canceling. It is widely used when data contain no outliers.

H. Mean Squared Logarithmic Error (MSLE)

Mean Squared Logarithmic Error (MSLE) measures squared differences after logarithmically transforming actual and predicted values. The transformation reduces emphasis on percentage differences involving large values and requires adding one because log(0) is undefined.

I. Root Mean Squared Logarithmic Error (RMSLE)

Root Mean Squared Logarithmic Error (RMSLE) applies logarithms to actual and predicted values before computing their difference. It is resistant to outliers and penalizes underestimation more than overestimation.

  • RMSLE is computed by applying logarithms to actual and predicted numbers and subtracting them.Source: §I · p. 6
  • RMSLE penalizes predictions below the actual value more than predictions above it.Source: §I · p. 6
  • RMSLE is useful when overestimation is acceptable but underestimation is unacceptable.Source: §I · p. 6
  • Normalized RMSE connects RMSE to the observed range, enabling comparisons between models with different scales.Source: J. Normalized Root Mean Squared Error… · p. 6

K. Relative Root Mean Squared Error (RRMSE)

RRMSE is a dimensionless RMSE variant that normalizes error against actual values, enabling comparisons across measurement approaches. Its reported score bands classify model accuracy from excellent to poor.

Definition

Interpretation

Equation

  • The RRMSE equation uses N data samples, true values y_i, and predicted values ŷ_i.The supplied passages identify these symbols but do not fully preserve the displayed equation.Source: K. Relative Root Mean Squared Error… · p. 7

Summary

M. LogCosh Loss

LogCosh computes the logarithm of the hyperbolic cosine of prediction error and combines smooth quadratic-like behavior with reduced sensitivity to large errors. The paper also situates it among the summarized loss functions and contrasts its role with quantile loss.

Definition

  • LogCosh computes the logarithm of the hyperbolic cosine of prediction error.It is described as smoother than quadratic loss and less affected by significant prediction errors.Source: M. LogCosh Loss · p. 7

Properties

  • The paper reports that LogCosh combines linear and quadratic scoring techniques and is extremely close to Huber loss.Source: M. LogCosh Loss · p. 7
  • LogCosh is twice differentiable, whereas the paper states that Huber loss can only be differentiated once.The text notes that some optimization approaches favor twice-differentiable functions.Source: M. LogCosh Loss · p. 8

Survey scope

Contrast with Quantile Loss

  • Quantile loss forecasts quantiles without assuming a parametric response distribution and can provide prediction intervals for non-constant variance residuals.The quantile parameter controls the balance between positive and negative errors.Source: N. Quantile Loss · p. 8

IV. EXPERIMENTS

The experiments evaluate forecasting loss functions across four datasets representing electricity consumption, traffic occupancy, grocery sales, and financial volatility. These datasets cover distinct temporal prediction settings and data types.

Datasets

  • The electricity dataset contains electricity consumption data from 370 points or clients.Source: A. Datasets • · p. 9
  • PEMS-SF contains 15 months of daily records describing occupancy rates across San Francisco Bay Area freeway lanes.The dataset contains 440 daily records and occupancy values range from 0 to 1.Source: A. Datasets • · p. 9
  • The Corporación Favorita dataset contains dates, store and item information, promotion status, and unit sales.Source: A. Datasets • · p. 9
  • The Volatility dataset contains daily non-parametric measures of historical volatility for financial assets or indexes.Source: A. Datasets • · p. 9

B. Performance Metrics

The experiments evaluate 14 loss functions on four datasets using P10, P50, and P90 metrics, with quantile forecasts providing upper and lower prediction bounds. Performance varies by dataset, although Quantile Loss appears among the better-performing losses in all four reported observations.

Evaluation setup

  • The experiment evaluates all loss functions using P10, P50, and P90 metrics.These correspond to the 10th percentile, median, and 90th percentile forecasts.Source: B. Performance Metrics · pp. 9–11

Quantile evaluation

  • Quantile forecasts at P10 and P90 define an 80% interval expected to contain the observed value 80% of the time.The interval is represented by the shaded region between P10 and P90.Source: B. Performance Metrics · pp. 9–10

Evaluation setup

  • The study uses Temporal Fusion Transformers with 1000 training samples, 100 validation samples, and 14 loss functions.The paper states that other losses were resolved into selected functions or were unsuitable for the listed datasets.Source: B. Performance Metrics · p. 11, Table XVII · p. 11

Dataset results

V. CONCLUSION

The paper frames loss-function selection as objective-dependent rather than universal, and summarizes practical choices for different data conditions. It presents 14 regression loss functions and proposes using the study as a baseline for time series forecasting applications.

  • Loss-function choice depends on outliers, data skewness, model requirements, computational constraints, and performance requirements.Source: §V · pp. 11–12
  • The study organizes cases in which particular loss functions may be useful, including datasets containing outliers.Source: §V · p. 12
  • Mean Squared Error is recommended with many outliers, whereas Mean Absolute Error is preferred when outliers are fewer.Source: §V · p. 12
  • LogCosh is suggested for maintaining balance, while Quantile loss is suggested when the objective is based on percentiles.Source: §V · p. 12
  • The paper summarizes 14 well-known loss functions and develops a tractable form intended to support improved and more accurate optimization.Source: §V · p. 12

Related papers

Loading 2211.02989v1…