A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification

• The framework targets classification when irregular sampling and variable sample counts prevent a natural fixed-dimensional feature representation, while sparse observations create uncertainty about underlying temporal processes.Source: §1 · p. 1
• The GP adapter uses Gaussian process regression to transform irregular time series into a uniform representation that preserves uncertainty and can feed any gradient-descent-learnable black-box classifier.Source: §1 · p. 1
• Exact GP regression costs O(n3) time and O(n2) space, making the adapter prohibitively expensive for large time series.Source: §1 · p. 2
• Combining SKI with Lanczos methods makes GP-adapter sampling linear in time and space with respect to series length and O(m log m) in time for m inducing points.Source: §1 · p. 2
• End-to-end discriminative training through the iterative Lanczos method outperforms baselines using GP mean interpolation or separately marginal-likelihood-trained GP regression across logistic, feedforward, convolutional, and MEG classifiers.Source: §1 · p. 2

• Each dataset case contains a time series and label, with series observed over a common interval but generally differing in observation times and numbers of samples.Source: §2 · p. 2
• Irregular sampling prevents common fixed-length-vector classifiers from directly applying, while sparse observations create substantial uncertainty about the underlying temporal processes.Source: §2 · p. 2
• GP regression represents each irregular series by its posterior marginal distribution at fixed reference points using a zero-mean prior, covariance kernel, and independent noise variance.Source: §2 · p. 2
• Exact posterior computation requires O(n3 + nd) time for the mean and O(n3 + n2d + nd2) time for the full covariance matrix.Source: §2 · p. 2

3.1 Classification frameworks and the Gaussian process adapter

• Expected loss propagates uncertainty from the GP posterior through the classifier and supports joint optimization of classifier and GP parameters.The expected loss is optimized using stochastic gradient descent and Monte Carlo approximation when needed.Source: §3.1 · p. 3, §3.2 · p. 3
• UAC trains with the expected loss over Gaussian representations, enabling predictions from the learned classifier and GP parameters on unseen time series.The GP parameters depend on both the time series and shared model parameters.Source: §3.1 · p. 3, §3.2 · p. 3
• The Gaussian process adapter maps each irregularly sampled time series to a common Gaussian representation that connects it to any gradient-based black-box classifier.The representation uses posterior mean and covariance at shared reference points.Source: §3 · p. 2, §3.1 · p. 3
• IMP replaces the random representation with its posterior mean, discarding uncertainty for simpler deterministic prediction and potentially suiting more densely sampled series.Training with UAC and predicting with IMP is also described as computationally efficient.Source: §3.1 · p. 3

3.2 Learning with the GP adapter

• The reparameterization z = µ + Rξ enables gradients through the Gaussian representation, while the symmetric square root R = Σ^1/2 is chosen for scalable approximation.The base classifier gradient is assumed to be efficiently computable.Source: §3.2 · p. 4

4.1 Structured kernel interpolation for approximating GP posterior means

• SKI approximates stationary kernel matrices with evenly spaced inducing points and sparse interpolation matrices, enabling fast kernel operations.Cubic interpolation uses at most four nonzero entries per interpolation-matrix row.Source: §4.1 · p. 4
• SKI-based conjugate-gradient computations approximate the posterior mean in O(n + d + m log m) time and O(n + d + m) space.Evenly spaced inducing points make the stationary kernel matrix Toeplitz and compatible with fast Fourier transforms.Source: §4.1 · p. 4

4.2 The Lanczos method for covariance square root-vector products

• Lanczos approximates Σ^1/2ξ in a Krylov subspace using only matrix-vector products, so neither Σ nor its square root must be formed explicitly.The approximation projects the square-root-vector product onto the subspace generated by ξ, Σξ, and subsequent iterates.Source: §4.2 · p. 5
• Posterior GP samples can be generated in O(n + d + m log m) time and O(n + d + m) space using SKI and Lanczos methods.This complexity applies when the number of Lanczos basis vectors k is O(1).Source: §4.2 · p. 5
• Block Lanczos and block conjugate gradients allow multiple posterior samples to share computations for estimating expected loss.The block methods operate on multiple random vectors associated with the same covariance matrix.Source: §4.2 · pp. 5–6

• End-to-end training jointly optimizes GP parameters θ and classifier parameters w instead of fixing GP representations after marginal-likelihood optimization.Source: §5 · p. 6
• The gradient of the posterior mean can be computed efficiently with fast Fourier transforms and conjugate gradients, matching the forward computation's time and space complexity.Source: §5 · p. 6
• Gradients through the covariance term are obtained by backpropagating through the differentiable Lanczos process and applying SKI techniques.Source: §5 · p. 6
• The gradient of H^1/2 is computed by solving a Sylvester equation with the Bartels–Stewart algorithm in O(k^3) time, treated as constant for small k.Source: §5 · p. 6
• Training the GP adapter takes O(n + d + m log m) time and O(n + d + m) space for m inducing points, n observations, and d generated features.Source: §5 · p. 6

• MEG is the closest related method and represents the uncertainty-aware kernel through random features that can be processed by the proposed algorithm.Source: §6 · p. 6
• SKI approximates the MEG random-feature computations with the same time and space complexity as the GP adapter.Source: §6 · p. 6
• The MEG random features form a stochastic classification layer with randomly initialized, non-trainable parameters.Source: §6 · p. 6
• Original MEG classification estimates GP parameters by marginal likelihood before classification, whereas the differentiable formulation supports efficient end-to-end training.Source: §6 · p. 6

7.1 Quality of GP sampling approximations

• Approximation error decreases as either the number of inducing points or Lanczos iterations increases, across output sizes of 1000, 2000, and 3000.Source: §7.1 · p. 7
• For a fixed normalized error across growing data sizes, both inducing points and Lanczos iterations must increase; k ≥3 is sufficient for estimating classification expected loss.Source: §7.1 · p. 7
• Lanczos approximation with SKI yields 1 to 3 orders of magnitude speed-ups for sampling and gradient computation versus exact computation.Source: §7.1 · p. 7

7.2 Classification with GP adapter

• The uWave experiment uses 3582 training and 896 test instances, with each series reduced by randomly sampling 10% of its 945 observations.Source: §7.2 · p. 8
• The UAC framework is tested with multi-class logistic regression, a fully connected network, and a convolutional network using joint stochastic optimization and early stopping.Source: §7.2 · p. 8
• Discriminative GP-parameter training always achieves better accuracy than marginal-likelihood training across the three classifiers and MEG, while posterior sampling beats posterior means.Source: §7.2 · p. 8, Table 1 · p. 8

8 Conclusions and future work

• The framework combines approximate GP sampling with classification for sparse, irregularly sampled time series and is validated against approximation, computational, and classification baselines.Source: §8 · p. 8
• Future work includes spectral mixture covariance functions [24], encoder LSTM classifiers [23], multidimensional time series and index sets, and autoencoder-based dimensionality reduction.Source: §8 · p. 8

A.1 Gradients of the approximate posterior GP covariance-vector product

• During Lanczos approximation, the vector d depends on θ and must therefore be handled correctly during backpropagation.Source: §A.1 · p. 10
• Auxiliary variables are introduced to reduce redundant computations in the gradient expressions.Source: §A.1 · p. 10, §A.2 · p. 11
• Gradients are provided for both the approximate posterior covariance-vector product and posterior mean with respect to GP hyperparameters and noise variance.The covariance-vector gradient is expressed using auxiliary variables as ∂eΣd/∂θ = Wx (β −ζ + η), while the posterior-mean gradient is ∂eµ/∂θ = Wx (β −γ).Source: §A.1 · p. 10, §A.2 · p. 11
• Cubic convolution interpolation is preferred over spline interpolation because spline-based SKI makes the interpolation matrix depend on GP parameters, requiring costly additional derivatives.The cubic convolution choice follows Keys. With spline interpolation, computing ∂W_a/∂θ introduces substantial backpropagation overhead.Source: B Cubic interpolation in backpropagation · p. 11

C Architectures used in the experiment

• The fully connected classifier consists of two layers, each containing 256 units.Source: C Architectures used in the experiment · p. 11
• The convolutional classifier has five layers: two one-dimensional convolutional layers, two size-2 max-pooling layers, and a final fully connected layer with 256 units.The convolutional layers use four filters of size 5.Source: C Architectures used in the experiment · p. 11