Source-linked AI summary
Learning from the past, predicting the statistics for the future, learning an evolving system
Daniel Levin, Terry Lyons, Hao Ni
TL;DR
The paper addresses non-parametric regression when inputs, and potentially outputs, are data streams whose sampled representations are difficult to use effectively. It uses graded path signatures as features for linear regression and applies the framework to stationary time series. The reported predictions have similar accuracy to Gaussian Processes while requiring much less computation, particularly in the numerical comparisons.
Problem
Non-parametric regression for streamed inputs and potentially streamed outputs requires features that capture stream effects despite sampling limitations and high-dimensional descriptions.
Method
The paper uses graded path signatures as feature sets and performs linear regression to estimate relationships between input streams and noisy output streams.
Results
The signature-based approach achieves similar accuracy to Gaussian Processes while using substantially less computation in the reported time-series comparisons.
Takeaways & Limitations
Signature regression offers an explicitly computable, broadly applicable approach for statistical inference on streamed data, including stationary time-series models.
Takeaways & Limitations
The expected-signature time-series model assumes stationarity, so its formulation is tied to time series whose joint distributions are invariant under time shifts.
Abstract
from arXiv · showhide
We bring the theory of rough paths to the study of non-parametric statistics on streamed data. We discuss the problem of regression where the input variable is a stream of information, and the dependent response is also (potentially) a stream. A certain graded feature set of a stream, known in the rough path literature as the signature, has a universality that allows formally, linear regression to be used to characterise the functional relationship between independent explanatory variables and the conditional distribution of the dependent response. This approach, via linear regression on the signature of the stream, is almost totally general, and yet it still allows explicit computation. The grading allows truncation of the feature set and so leads to an efficient local description for streams (rough paths). In the statistical context this method offers potentially significant, even transformational dimension reduction. By way of illustration, our approach is applied to stationary time series including the familiar AR model and ARCH model. In the numerical examples we examined, our predictions achieve similar accuracy to the Gaussian Process (GP) approach with much lower computational cost especially when the sample size is large.
1 Introduction
The paper applies rough-path signatures to non-parametric regression on streamed inputs and potentially streamed outputs, addressing the difficulty of extracting predictive features from highly sampled data. It presents a broadly applicable signature-based framework and reports comparable accuracy to Gaussian Processes with substantially lower computational cost.
- Motivation for signatures as a feature set: Sampling can miss streams with different effects despite near-identical sampled values, while high-dimensional stream descriptions make regression feature identification difficult.Fourier series and wavelets are also described as no more effective than sampling because they are linear path functionals.
- Results: The practical tests report similar accuracy to Gaussian Process methods but many-factors-better computational time for the signature-based approach.The paper frames the comparison as especially relevant for robust and efficient forecasting of streamed-data effects.
- Applications: The framework is applied to stationary time-series models, including ARX or NARX settings, and benchmarked against classical and non-parametric approaches.The experiments use data generated from linear or non-linear time-series models and compare against AR calibration and Gaussian Processes.
- Signatures: The signature summarizes stream behavior through progressively more order-sensitive terms, capturing event order without directly engaging the stream’s micro-structure.The signature can be truncated to provide graded descriptions of streams, while higher-order terms retain increasingly noncommutative information.
- Signatures: Expected signatures provide a non-parametric description of the probability measure on stream space, with grading enabling approximate descriptions of stream laws.This connects signature features with statistical inference beyond individual stream representation.
- The expected signature model: The paper introduces linear regression on signature features to estimate functional relationships between input streams and noisy output streams in a general non-parametric framework.The approach is motivated by the signature’s role as a feature set and is presented with calibration and forecasting methods.
2 Preliminaries
The preliminaries define path signatures and rough-path concepts, explain why signatures arise from controlled differential equations, and develop properties supporting computation and statistical use. Key properties include truncation, concatenation, time-reparameterization invariance, algebraic identities, and uniqueness results.
- Signatures: A path signature is a formal power series whose coefficients are iterated integrals, with truncated signatures retaining terms through a chosen order.The signature lies in a tensor algebra, and truncation provides a finite-order representation.
- The origin of signatures: Signatures arise naturally from Picard iteration for controlled differential equations, whose finite iterations are linear functionals of truncated driver signatures.The limiting solution motivates using signatures to represent the effects of driving paths.
- Properties of signatures: Chen’s identity makes the signature multiplicative under path concatenation, supporting computation for piecewise-linear paths.The signature of a concatenated path can be assembled from signatures of its constituent segments.
- Properties of signatures: The signature is invariant under time re-parameterization, while its higher-order terms retain information about the ordering of events along a path.This combines the formal invariance property with the stream-summary interpretation developed in the introduction.
- Properties of signatures: The signature determines finite-variation paths up to tree-like equivalence, and Brownian paths are determined up to time re-parameterization almost surely.A monotone coordinate gives a sufficient condition for unique path determination, while the Brownian result concerns Stratonovich signatures.
3 A general expected signature framework
The expected signature framework models streamed-input regression by replacing path features with signatures, whose linear functionals can approximate continuous relationships and whose truncations enable finite-dimensional computation. In examples, truncated signatures improve prediction while reducing dimensionality relative to increment features.
- 3.1 The expected signature model: Linear regression on path signatures provides a general approach to estimating unknown functional relationships between input and potentially streamed output variables.The framework targets conditional expectations or distributions of output streams given input streams.
- 3.1 The expected signature model: Signature linear functionals can approximate continuous functions on compact sets of signatures to arbitrary precision.The result follows from the algebraic properties of signatures and the Stone-Weierstrass theorem.
- 3.1 The expected signature model: The expected signature model adds zero-conditional-mean noise and represents conditional output behavior through a linear functional of the input signature.The model is calibrated by regressing truncated output-signature coordinates on truncated input-signature coordinates.
- 3.1 The expected signature model: Truncating signatures converts the infinite feature representation into a finite regression problem while retaining an efficient summary of the stream.For a two-dimensional path, an order-n truncated signature has dimension 2^(n+1) − 1, independent of the number of sampling points K.
- 3.3 An illustrative example: a diffusion process: Increasing signature order improves fitting: an order-6 truncated signature gives almost perfect prediction, while an order-4 signature with dimension 31 outperforms 250 increment features.The example reports stronger prediction and dimension reduction relative to the increment-feature representation.
- 3.3 An illustrative example: a diffusion process: Increment features become more prone to overfitting as sampling density increases, whereas truncated signatures keep dimension independent of the number of sampling points.Regularization can help increment features, but the comparison still favors the signature approach in the reported example.
4 Time series models
The section embeds time series into continuous paths and computes signatures as compact features for regression, linking the resulting ES model to classical time-series models.
- 4.1 The signature of a time series: The time-series signature is computed by embedding observations into a continuous path and then calculating that path’s signature.The procedure explicitly consists of path embedding followed by signature computation.
- 4.1 The signature of a time series: The time-joined path is preferred for connecting the signature model with classical time-series models, whereas lead-lag paths expose volatility through second-level signature terms.Piecewise linear interpolation may lose information in the univariate case because it retains only the terminal increment.
- 4.1.1 The transformation by retaining the time component: The time-joined transformation retains each return between observations and introduces an instantaneous jump when new data arrive.A zero point is added at the initial time to define the transformed series.
- 4.3 The link between the classical time series models and the ES model: The ES model uses signatures of past returns to model future-return signatures and includes classical AR, ARCH, and GARCH models as special cases.The ARCH connection is formalized by showing that a sufficiently large ES model represents the ARCH setting.
5 Examples and numerical computation
The experiments compare AR, Gaussian Process, and expected-signature regression on simulated time series, using cross-validation and predictive-error, fit, and runtime measures. Expected-signature regression achieves comparable accuracy to GP while requiring much less computation, and is more robust than AR under model misspecification.
- Methods: The ES implementation uses linear regression of the next return against the truncated signature of the p-lagged values, alongside AR and GP baselines.The GP implementation uses exact inference with a squared exponential covariance function.
- Evaluation: Predictive evaluation uses repeated random sub-sampling cross-validation with R2, adjusted-R2, MSE against the true conditional mean, and running time.The reported results include 20-fold cross-validation and summaries of computational time.
- Comparison with AR: The ES approach outperforms AR on r(2) and r(3), while AR performs slightly better on r(1) by MSE.Using R2 and adjusted-R2, ES is equal to AR on r(1) and better on the other two datasets.
- Comparison with GP: Expected-signature regression produces comparable fitting results to GP while taking much less computational time, including computation below one thirty-second of GP in the reported comparison.The cross-validation results also describe both methods as more robust than AR, especially under model misspecification.
A Rough Paths
Rough path theory represents evolving systems through continuous, finite-variation path objects and their tensor-valued extensions. Its extension theorem uniquely lifts suitable multiplicative functionals to higher degrees, while Chen’s identity supplies the relevant algebraic structure.
- Definitions: A p-rough path is a continuous map into a truncated tensor space satisfying multiplicativity and finite p-variation conditions.The space of p-rough paths in E is denoted Ωp(E).
- Algebraic structure: Equation (A.1) is derived from Chen’s identity for signatures.
- Definitions: A geometric p-rough path is a p-rough path approximable by 1-rough paths in the p-variation distance.The corresponding space is denoted GΩp(E).
- Extension theorem: The extension theorem uniquely extends a finite-p-variation multiplicative functional to arbitrary higher degree when its initial degree is at least floor(p).This theorem is identified as a main ingredient of rough paths theory.
B Time series models
The paper defines standard autoregressive and ARCH time-series models through lagged returns, white-noise errors, and conditional variance dynamics. These definitions establish the parametric models used for comparison and mean-process specification.
- AR model: An AR(p) model expresses the current return as an intercept plus p lagged returns and zero-mean white noise.The parameters are Φ = [Φ0, . . . , Φp].
- ARCH model: The ARCH mean equation is a linear combination of lagged returns.
C.1 The Poly AR model
The Poly AR model is introduced as one of the simulated time-series models, with its calibration parameters estimated from the second dataset. The supplied passages identify the coefficient-estimation setup but do not include the resulting coefficient values.
- Calibration: The second dataset’s AR calibration estimates the parameter vector Φ, while the Poly AR specification uses n = 4 and enumerates indices with corresponding coefficients.
C.2 The Mixture of Poly ARs model
The section reports parameter estimates from AR calibration, the ES approach, and the GP approach for the third data set.
- The AR calibration provides an estimator for Φ for the third data set.
- For n = 4, the ES approach yields inferred parameters for the model.
- The GP approach fits ln(λ) = 2.8852 and ln(h) = 1.9846.