PaperlayerView exact revision on arXiv

Source-linked AI summary

Trend Filtering for Functional Data

Tomoya Wakayama, Shonosuke Sugasawa

arXiv:2104.02456v3stat.MEstat.CO

TL;DR

Flexible estimation of functional trends, especially with abrupt changes, remains limited. This paper extends trend filtering to functional time series and graph-indexed data, using penalties on adjacent-function differences and an efficient optimization procedure. The authors report superiority to existing methods in simulation and empirical studies, and improved simulation performance when selecting the number of basis functions.

  • Problem

    Flexible estimation of functional trends remains limited, and handling abrupt trend changes is challenging.

    Source: §1 · p. 2

  • Method

    The paper extends trend filtering to functional time series and graph-indexed data, penalizing L2-norms of adjacent-function differences and using an orthonormal-basis expansion with an iterative optimization algorithm.

    Source: §1 · pp. 2–3

  • Results

    The proposed method was superior to existing methods in simulation and empirical studies; in simulation, selecting the number of basis functions improved performance.

    Source: §6 · p. 23

  • Takeaways & Limitations

    Selecting basis functions by penalizing redundant ones can improve simulation performance, while reducing unnecessary basis functions denoises the functions themselves.

    Source: §6 · p. 23

    The functional setting assumes square-integrable functions on a compact set X ⊂ R.

    Source: §3.1 · p. 5

Abstract

from arXiv · show

Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulate the new trend filtering by introducing penalty terms based on $L_2$-norm of the differences of adjacent trend functions. We develop an efficient iteration algorithm for optimizing the objective function obtained by orthonormal basis expansion. Furthermore, we introduce additional penalty terms to eliminate redundant basis functions, which leads to automatic adaptation of the number of basis functions. The tuning parameter in the proposed method is selected via cross validation. We demonstrate the proposed method through simulation studies and applications to real world datasets.

1 Introduction

Effective estimation of functional trends remains underdeveloped, particularly for non-stationary time series and spatial data. The paper extends trend filtering to functional observations to support locally adaptive smoothing.

1 Introduction

  • Flexible estimation of functional trends is limited for non-stationary time series and spatial data.Existing approaches for functional time series have limited flexibility for trend estimation, while many flexible methods for spatially varying trends have not been considered.Source: §1 · p. 2
  • Abrupt trend changes challenge existing functional methods, motivating locally adaptive smoothing.Scalar trend filtering is recognized as a powerful locally adaptive approach, and its spatial extension estimates trends with abrupt changes.Source: §1 · p. 2
  • The paper extends scalar trend filtering to functional time series using L2-loss and L2-norm penalties on differences between adjacent trend functions.The resulting objective defines functional trend filtering.Source: §1 · pp. 2–3
  • An orthonormal-basis expansion yields a fused-lasso and group-lasso objective, optimized by an ADMM-based iterative algorithm.The method also extends to graph-indexed data and uses cross-validation for tuning-parameter selection.Source: §1 · p. 3

2 Review of trend filtering for scalar data

The review presents scalar trend filtering as a discrete-difference approach to estimating trends, then describes its extension to graph-indexed spatial data.

2 Review of trend filtering for scalar data

  • Scalar kth-order trend filtering estimates an underlying trend by minimizing an objective involving a discrete difference operator.The tuning parameter λ controls the trade-off between fit and regularization.Source: §2 · pp. 3–4
  • Sparse kth-order differences yield piecewise-polynomial trend estimates; at k = 0, the method has fused-lasso form and produces piecewise-constant estimates.The review notes that the convex regularization problem remains computationally efficient.Source: §2 · p. 4
  • For spatial data, sample locations are graph vertices and adjacency relationships are edges, enabling trend filtering through graph differences.Sparsity in graph differences yields a piecewise-polynomial estimator, as in scalar trend filtering.Source: §2 · pp. 4–5

3 Functional trend filtering

The section formulates functional trend filtering for time-indexed mean functions and extends it to graph-indexed functions. Orthonormal basis expansion supports optimization, while an additional group penalty allows basis selection.

3 Functional trend filtering

  • The target is the sequence of mean functions β_t=E[Y_t], estimated by penalizing kth-order differences between adjacent trend functions.Source: §3.1 · pp. 5–6

3.1 Settings and objective function

  • After orthonormal basis expansion, the functional objective has a penalty mixing fused-lasso and group-lasso structure, unlike scalar trend filtering’s fused-lasso-like penalty.Source: §3.1 · p. 6

3.2 Optimization

  • The authors formulate optimization as an ADMM-like problem and develop an iterative algorithm, whose convergence they confirm empirically.Source: §3.2 · p. 7
  • The functional HP alternative uses a squared L2-norm penalty, is easy to implement, but does not produce sparsity in the differences.Source: §3.2 · p. 8
  • K-fold cross-validation selects λ by withholding every Kth function and estimating its trend from the midpoint of its smoothed neighbors.Source: §3.2 · p. 8

3.3 Extension to functional data on graph

  • For graph-indexed functions, the proposed kth-order trend filter penalizes local variation measured by graph differences and can be solved using Algorithm 1.Source: §3.3 · p. 10

tion

  • A group-lasso penalty treats each basis function’s coefficients as a group, zeroing unnecessary groups so the number of retained basis functions adapts to the data.The authors select many basis functions initially and use the survivors as essential bases.Source: tion · p. 11

4 Simulation Studies

The simulations compare functional trend filtering with existing methods across four trend scenarios, then test whether sparse functional trend filtering can select basis functions. FTF generally performs better than the alternatives, while SFTF outperforms oracle-tuned DFPC in the reported comparison.

4.1 Procedure

  • The four scenarios test recovery of a horizontal line, a continuous curve, discontinuities between horizontal segments, and a sharp peak with changing smoothness.The trends span t = 1,...,50 and x in [1,120], with observations generated by adding Gaussian noise at 120 equally spaced domain points.Source: §4.1 · pp. 13–14, Figure 1 · p. 14

4.2 Functional trend filtering

  • The comparison used FTF and FHP with k in {0, 1, 2} and FPC; FPC-derived basis functions were shared by FTF and FHP with L = 5.This setup was intended to make the comparison independent of basis functions, and tuning parameters were selected by 10-fold cross-validation.Source: §4.2 · p. 15
  • FTF tended to outperform FHP and FPC overall, while Figure 2 shows FPC's estimates were under-smoothed relative to FTF and FHP.The comparison used MSE over 150 repeated simulations.Source: Figure 2 · p. 15, Table 1 · p. 16, §4.2 · p. 17
  • In scenario 3, FTF outperformed FHP and FPC; FTF with k = 0 was most accurate there, while k = 1 performed especially well in scenario 4.FTF was slightly inferior to FHP in scenarios 1 and 2; the authors suggest iterative approximation versus an analytical solution as one possible reason.Source: §4.2 · p. 17

4.3 Sparse functional trend filtering

  • With parameters chosen to favor DFPC, SFTF nevertheless dominated it and retained a fairly smaller than 10 number of nonzero-coefficient basis functions; in scenarios 2 and 4, selected components were far fewer than 5.The authors state that excluding redundant basis functions plays a critical role in increasing accuracy.Source: §4.3 · p. 19

5 Applications

The applications show functional trend filtering smoothing Australian fertility curves while retaining changes, and localizing spikes in Japanese prefecture COVID-19 data. The fertility analysis also identifies recurring structural changes in the trend-filtered scores.

5.1 Australian fertility rates

  • The fertility functions cover ages 15–49 from 1921 to 2015, while the prefecture data track infections per million from January 16, 2020 to March 9, 2021.Source: §5.1 · p. 19, §5.2 · p. 22
  • Compared with FPC, SFTF serves as a smoother for births per 1,000 females at ages 20 and 30, eliminating small noises while retaining significant change points.Source: §5.1 · p. 20
  • Changes around 1945, 1960–62, and 1970–72 emerge in the trend-filtered scores, with detected points tending to overlap for k = 1 and 2.The original principal-component scores did not reveal the same structural properties.Source: §5.1 · pp. 21–22, Figure 5 · p. 22

5.2 The number of COVID-19 cases in Japanese prefectures

  • On day 395, FTF localized estimates around strong inhomogeneous spikes and highlighted Tokyo, while FPC results remained jagged.The displayed FTF fit uses k = 2; the k = 1 result was almost the same.Source: §5.2 · p. 23

6 Discussion

The discussion presents functional trend filtering as a locally adaptive smoothing method for time-series and spatial functional data. It also describes basis-function selection as a way to denoise time-series functions and improve simulation performance.

6 Discussion

  • The authors report that the proposed method outperformed existing methods in simulation and empirical studies.Source: §6 · p. 23
  • In time-series data, a penalty selects the number of basis functions; removing unnecessary bases denoises functions, while trend filtering smooths across time.The authors report improved simulation performance from selecting the number of bases rather than simply using more.Source: §6 · p. 23
  • The authors suggest applying the grouped- and fused-lasso optimization techniques to other statistical problems, including regression with complicated sparsity-inducing penalties.Source: §6 · p. 23
  • Figure 6 contrasts observed day-395 prefecture cases with FPC-smoothed and second-order FTF-smoothed values.Source: Figure 6 · p. 24, §6 · p. 24

Appendix 1: Derivation of Algorithm 1

The appendix derives the updates used in Algorithm 1, including a proximal FISTA step for the nonsmooth update.

Appendix 1: Derivation of Algorithm 1

Appendix 2: Algorithm of sparse functional trend fil-

The sparse functional trend-filtering algorithm changes the bℓ update used in Algorithm 1.

Appendix 2: Algorithm of sparse functional trend fil-

  • For sparse functional trend filtering, the bℓ update is computed with FISTA, following the approach used for the a_t update in Algorithm 1.Source: tering · p. 26
Loading 2104.02456v3…