PaperlayerView exact revision on arXiv

Source-linked AI summary

SCORE: Spectral Correlation Estimation for Multivariate Gaussians

Christopher Bülte, Emil Partow, Astha Gupta, Pascal Esser, Gitta Kutyniok

arXiv:2610.12096v1cs.LG

TL;DR

High-dimensional Gaussian prediction needs covariance approximations that retain dependencies while remaining computationally efficient and numerically stable. SCORE combines kernel-score training with staged marginal and spectral-correlation estimation. The paper reports improved performance across evaluated tasks and additive finite-sample error bounds for the two stages.

  • Problem

    High-dimensional Gaussian predictions need expressive covariance models, but dense matrices are costly, structured alternatives restrict correlations, and likelihood training is unstable near singular covariance.

    Source: §1 · p. 1

  • Method

    SCORE fits marginal distributions first, then learns a structured correlation in spectral space using the closed-form Gaussian kernel score.

    Source: §1 · pp. 1–2

  • Results

    SCORE outperforms established baselines across a range of metrics and is the best calibrated among the compared methods.

    Source: §6 · p. 8

  • Takeaways & Limitations

    A Fourier-domain diagonal core produces dense circulant dependencies with linear storage and FFT-based O(d log d) operations.

    Source: §1 · p. 2

    The implied circulant covariance may be unreasonable for some applications, which may require richer structures.

    Source: §6 · p. 8

Abstract

from arXiv · show

Neural network-based predictive modeling with high-dimensional structured Gaussian targets requires an efficient and numerically stable, yet expressive approximation of the covariance matrix. We propose SCORE: a scalable framework, combining scoring rule training with an expressive covariance approximation learned in spectral space. For $d$-dimensional data, the learning task is decomposed into learning the marginal distributions and learning a structured correlation matrix, which enables dense dependencies with linear storage and $\mathcal{O}(d\log d)$ cost. We utilize the closed form Gaussian kernel score for training, which remains defined even for degenerate covariances and admits bounded gradients during optimization. We characterize kernel scores under invertible transforms and prove exact invariance under unitary transforms. At population level, our two-level objective recovers the true marginals and projects the target correlation onto the representable class; finite-sample PAC bounds show that the errors of the two stages enter additively. We evaluate our model on a variety of tasks with a commonly assumed Gaussian domain: Time-series forecasting, monocular depth estimation, and spatial weather prediction, showing improved performance at lower computational cost.

1 INTRODUCTION

High-dimensional Gaussian prediction needs covariance models that preserve structured dependencies without dense-matrix cost or unstable likelihood optimization. SCORE pairs a robust kernel-score objective with staged marginal and spectral-correlation estimation.

1 INTRODUCTION

  • Dense covariances require O(d^2) parameters and up to O(d^3) operations, while diagonal and low-rank alternatives restrict representable correlations.The negative log-likelihood is undefined for singular covariances and can have arbitrarily large gradients near the positive-definite boundary.Source: §1 · p. 1
  • SCORE combines closed-form Gaussian kernel-score training with a covariance approximation that separates marginal uncertainty from dependence structure.The kernel score remains defined for degenerate distributions and has bounded gradients; the method uses unitary-transform invariance to learn dependence spectrally.Source: §1 · pp. 1–2
  • Stage 1 estimates marginal distributions and standardizes the target; Stage 2 fits the remaining correlation structure in a spectral domain.Rescaling the learned correlation by the marginal scales gives the approximate predictive covariance.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • A Fourier-domain diagonal core yields a dense circulant correlation matrix with linear storage and FFT-based O(d log d) operations.At population level the objective recovers true marginals and selects the best representable correlation; finite-sample PAC bounds make the two stages' errors additive.Source: §1 · p. 2
  • The framework is evaluated on real-world high-dimensional spatiotemporal prediction tasks, examining performance, calibration, numerical stability, and computational efficiency.The listed application domains include atmospheric forecasting, oceanographic fields, biological measurements, and astronomical time series.Source: §1 · pp. 1–2

2 BACKGROUND

The background defines scoring-rule training and surveys covariance parameterizations before motivating alternatives to likelihood optimization for high-dimensional Gaussian predictions.

2.1 SCORING RULE MINIMIZATION

  • A scoring rule assigns a score to a probabilistic prediction and observation; strict propriety makes expected score minimization recover the target distribution.The log score is a strictly proper example whose associated divergence is KL, recovering cross-entropy minimization.Source: §2.1 · pp. 2–3

2.2 PARAMETERIZING A MULTIVARIATE GAUSSIAN

  • Diagonal covariance discards correlations, Cholesky is fully expressive but computationally infeasible at high dimensions, and low-rank-plus-diagonal offers a middle ground.The low-rank-plus-diagonal form is Σ = D + UU⊤ with rank r much smaller than d.Source: §2.2 · p. 3
  • The covariance comparison table summarizes parameter counts, log-score evaluation, sampling, and covariance-entry estimation costs.The section also notes that Kronecker approximations factor structured domains but still require an approximation for each factor.Source: Table 1 · p. 3, §2.2 · p. 3

2.3 TRAINING WITH NEGATIVE LOG-LIKELIHOOD

  • The log score requires full-rank covariance and is undefined for rank-deficient targets, while its gradients become unbounded near singularity.Adding εI can still cause numerical issues for small ε and overestimate aleatoric uncertainty for large ε.Source: §2.3 · p. 3

3 METHODOLOGY

The methodology replaces unstable likelihood training with a kernel score and uses unitary spectral transforms to model covariance efficiently. SCORE separates marginal fitting from correlation fitting, with population and finite-sample guarantees for the staged objective.

3.1 ROBUST LOSS FUNCTIONS

  • For Gaussian predictions, the closed-form Gaussian kernel score has bounded gradients and is globally Lipschitz, unlike the log score's singular behavior.In the low-rank toy example, the figure reports that the log score fails to recover the posterior while the kernel score does.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 SPECTRAL PARAMETERIZATION

  • Kernel scores transform consistently under invertible maps and are exactly invariant under unitary transforms, allowing correlation fitting in a spectral domain without changing the objective.With the DFT and a diagonal spectral covariance, the inverse transform is a dense circulant covariance, and FFT operations cost O(d log d).Source: §3.2 · p. 4

3.3 SPECTRAL CORRELATION ESTIMATION

  • SCORE separates marginal scales from correlation because a circulant covariance alone forces equal marginal variances.The parameterization applies the correlation operator to the transformed core so the predicted marginal variances equal the learned τ values.Source: §3.3 · pp. 4–5
  • The staged objective fits marginals first, freezes them, then fits the correlation core; a joint fit would distort marginals to improve joint score and lacks an O(d) kernel-score form.For the DFT and diagonal core, a normalized spectrum defines the correlation core; the implementation uses CRPS for the first stage and Gaussian kernel score for the second.Source: §3.3 · p. 5
  • At population level, strictly proper first- and second-stage scores recover true marginals and select the closest representable correlation under the associated divergence.The deployed prediction retains the true mean and marginal scales while using the representable correlation.Source: §3.3 · pp. 5–6
  • Finite-sample PAC bounds show that the two stages' errors enter additively rather than having the second stage amplify errors in estimated marginals.The guarantee is stated for CRPS in stage one and Gaussian kernel score in stage two, under the paper's assumptions.Source: §3.3 · p. 6
  • The marginal-exactness result and population optimum extend to any unitary transform and admissible core class, although the quantitative results mainly concern Fourier diagonal cores.For the log score, closed forms extend to arbitrary transforms and low-rank cores; the Gaussian kernel score does not have that same extension.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

Across eleven datasets, SCORE ranks best on most reported metrics, is the fastest dependence-modeling baseline, and is best calibrated overall except on NYU. The experiments compare Gaussian predictive methods under shared backbones and evaluate predictive performance, calibration, and compute.

Datasets & baselines

  • The study compares methods across four Gaussian-predictive tasks using shared backbones, with Fourier transforms except for the station task’s graph Fourier transform.Evaluation uses test-set averages over five seeds and metrics covering mean, marginal fit, joint distribution, and dependence; training objectives overlap with several evaluation metrics.Source: §4 · p. 6

Results

  • SCORE has the best average rank (2.5) and ranks best on MSE, CRPS, ES, and VS, ahead of LorD-KS (3.1) at equal parameter count.The average ranks are over eleven datasets.Source: §4 · p. 6, Table 3 · p. 7
  • Kernel-score training beats NLL on the large majority of non-objective metric comparisons, while SCORE ranks best among KS-trained models on every metric, especially VS.The win-count comparison is significant under a data-set-clustered sign-flip test; NLL is won by LorD-NLL, while SCORE ranks similarly to the KS-trained model (±0.2) on KS.Source: §4 · p. 7, Table 4 · p. 7
  • Against Diag, SCORE is the fastest dependence-modeling baseline and performs best, with clear margins except on MSE, where it is similar to LorD.Figure 4 plots median performance improvement against increased compute across seeds and datasets.Source: Figure 4 · p. 7, §4 · p. 7
  • SCORE is best calibrated overall, except on NYU, where depth maps violate the stationarity assumption and SCORE falls behind.Figure 5 compares calibration measures for LorD, SB, and SCORE; its dashed diagonal marks optimal calibration.Source: §4 · p. 7, Figure 5 · p. 8

5 RELATED WORK

Prior work has used neural networks to predict Gaussian distributions with diagonal, full, and structured covariance outputs, while Gaussian processes provide a distinct kernel-based route. SCORE follows direct conditional distributional regression rather than modeling posterior uncertainty about a latent function.

Neural Gaussian prediction

  • Mean-variance networks output univariate Gaussians, while multivariate extensions use diagonal covariances; NLL gradient difficulties motivated reweighted losses and alternative parameterizations [12] [40].Source: §5 · p. 8

Covariance parameterizations

  • Full covariance prediction via Cholesky factors and lower-cost alternatives such as banded Cholesky, Taylor expansions, and low-rank-plus-diagonal structures have been explored [3] [41] [19].Low-rank-plus-diagonal methods also appear in image uncertainty and time-series prediction [21] [4].Source: §5 · p. 8

Gaussian processes

  • Gaussian processes induce input-dependent means and covariances from kernels, but exact inference costs O(d^2) memory and O(d^3) time; SCORE instead predicts the conditional law directly [45, 46, 47, 48, 49, 50, 51, 1, 52].The direct distributional-regression approach describes aleatoric dependence rather than posterior uncertainty about a latent function.Source: §5 · p. 8

6 CONCLUSION

SCORE combines structured Gaussian prediction with staged marginal and correlation learning. Its conclusion highlights benchmark performance and computational trade-offs, while the supplied appendix passages set notation and describe the log score.

6 CONCLUSION

  • SCORE outperforms established baselines across metrics, is best calibrated among compared methods, and offers the best compute–performance trade-off.The circulant covariance structure may be unreasonable in some applications; richer transformed-domain parameterizations and other unitary transforms are suggested for future work.Source: §6 · p. 8
  • SCORE’s main limitation is its implied circulant covariance structure, which may be unreasonable for some predictive tasks.The paper suggests richer transformed-domain parameterizations, such as low-rank or tridiagonal cores, and studying other unitary transforms such as wavelets.Source: §6 · p. 8

A NOTATION

  • The notation defines the training data as i.i.d. input–output samples and the predictive model as a neural network mapping inputs to probability distributions.Unless stated otherwise, predictions are Gaussian with a mean vector and positive-definite covariance; hats denote transformed coordinates under a unitary matrix.Source: A NOTATION · p. 14

B.1 LOG SCORE

  • The multivariate Gaussian log score is expressed with log det Σ, a quadratic form in (y − µ) weighted by Σ−1, and a dimension-dependent log(2π) term; gradients use w and z.The passage defines w := Σ−1r and z := L−1r, with L⊤w = z.Source: §B.1 · pp. 14–15

B.1.1 Proof of Lemma 2.1

The appendix establishes instability of log-score gradients near singular covariances, bounded gradients for the Gaussian kernel score, and properties of transformed scores and normalized covariance cores.

B.1.1 Proof of Lemma 2.1

  • The log-score gradient is unbounded near the positive-definite boundary along a covariance sequence whose smallest eigenvalue approaches zero.The proof also gives a sequence showing unbounded gradients even for a fixed residual component equal to zero.Source: §B.1.1 · p. 15

B.2.1 Proof of Theorem 3.1

  • The Gaussian kernel score has bounded gradients and is globally Lipschitz in the mean and Cholesky factor, including when the factor is singular.The bounds do not require invertibility of the lower-triangular factor.Source: §B.2 · p. 15, §B.2.1 · p. 17

C.1.1 Proof of Proposition 3.2

  • An invertible transform preserves strict propriety of the transformed kernel score, while a unitary transform leaves the score unchanged.The proof uses characteristicness of the kernel for strict propriety and B*B = I for unitary invariance.Source: §C.1.1 · p. 18

C.1.2 Proof of Lemma 3.3

  • Normalizing a positive-definite transformed covariance by its diagonal yields a well-defined positive-definite correlation matrix with unit diagonal.The proof establishes this through congruence by an invertible diagonal scaling matrix.Source: §C.1.2 · p. 18

C.1.3 Proof of Proposition 3.4

The proposition’s proof separates exact marginal fitting from correlation approximation. The implementation realizes structured covariance predictions through spectral cores and closed-form scoring computations.

C.1.3 Proof of Proposition 3.4

  • The first-stage objective recovers each true Gaussian marginal, while the second stage selects the representable correlation minimizing divergence from the target correlation.The deployed prediction retains the target marginal variances.Source: §C.1.3 · p. 18

C.2.1 From network outputs to a prediction

  • For a diagonal core, the network head emits mean, marginal-scale, and spectrum parameters, with softplus and softmax mappings enforcing positive variances and a normalized positive spectrum.The spectrum is normalized to sum to d; for the DFT it is symmetrized so the resulting covariance is real.Source: §C.2.1 · p. 19
  • A low-rank correction adds r directions beyond those represented by the transformed diagonal core; for the DFT, these are non-stationary directions over a stationary background.The transformed core becomes diag(s) + U-hat U-hat*.Source: §C.2.1 · p. 19

C.2.2 Scores and closed forms

  • The first stage uses univariate Gaussian CRPS, while the second scores the standardized residual and uses different closed forms for the log and kernel scores.The kernel-score computation in the second stage scores the core because standardization is not unitary.Source: §C.2.2 · pp. 19–20
  • For a DFT diagonal core, the Whittle likelihood is exact because the back-transformed covariance is circulant by construction.The passage contrasts this with the usual approximation for stationary Gaussian time series, which replaces a Toeplitz covariance with a circulant one.Source: §C.2.2 · p. 20
  • With the FFT, evaluating either objective costs O(d log d) and uses O(d) memory for the spectrum and marginals.For low-rank cores, the log score retains closed forms, with additional costs for transforms, assembling an r × r matrix, and factorization; the kernel score does not extend likewise.Source: §C.2.2 · p. 20
  • In standardized coordinates, the median-heuristic bandwidth is γ^2 = 4d, independently of the correlation matrix, target, and input.The implementation averages the second-stage loss over a ladder of bandwidths around this value and evaluates at γ^2 = d.Source: §C.2.2 · p. 21

D EXPRESSIVITY OF SCORE

The section characterizes how diagonal-core SCORE approximates target correlations under kernel and log scores, and develops a two-stage PAC analysis of marginal estimation and correlation learning in standardized coordinates. It shows how marginal errors combine with estimation and approximation error in the second-stage guarantee.

D EXPRESSIVITY OF SCORE

  • Under the kernel score, approximation error is equivalent up to constants to Frobenius projection error, which in the diagonal case is Fourier-domain off-diagonal energy.The errors vanish together; at the training bandwidth, the stated lower constant can be extremely small, so the equivalence is qualitative.Source: D EXPRESSIVITY OF SCORE · pp. 21–22
  • Under the log score, diagonal-core approximation error is -1/2 log GM of the transformed normalized spectrum and vanishes if and only if F diagonalizes RQ.The kernel and log scores agree on when the error vanishes but differ in how they scale the remaining error.Source: D EXPRESSIVITY OF SCORE · p. 23

E.1 SETTING AND ASSUMPTIONS

  • The PAC analysis assumes exact empirical risk minimization and disjoint stage samples, with a DFT diagonal core, CRPS for marginals, and Gaussian kernel score for correlations.The setting bounds target means and scales and uses kernel bandwidth γ̃^2 = 4d; admissible spectra define real correlation cores.Source: E PAC LEARNABILITY · p. 23, §E.1 · p. 24

E.2 THE FIRST STAGE IS PAC

  • Concentration bounds excess risk, while CRPS curvature converts that bound into parameter error, establishing the first-stage PAC argument.The analysis introduces the curvature lemma for normally distributed outcomes.Source: §E.2 · p. 26
  • The first-stage empirical risk minimizer has PAC bounds for marginal errors in both original and standardized coordinates under Assumptions E.1–E.2.The proof uses uniform deviation to control excess risk and CRPS curvature to convert it into parameter error.Source: §E.2 · pp. 27, 29

E.3 THE SECOND STAGE IN STANDARDIZED COORDINATES IS PAC

  • First-stage errors affect the standardized second-stage objective linearly and independently of the model class and sample size, so the second stage does not amplify them.The target-replacement bound is uniform over spectra; at bandwidth 4d, its dimension dependence cancels.Source: §E.3 · pp. 29–30
  • The standardized-coordinate PAC guarantee holds with probability at least 1 − δ1 − δ2 and combines first-stage error, second-stage estimation error, and approximation error.The second-stage deviation is controlled using a finite net and Hoeffding’s inequality.Source: §E.3 · pp. 30–31

E.4 SCORE IS PAC

The original-coordinate PAC bound combines approximation and finite-sample errors. Its proof separates correlation error from marginal error and uses a refined comparison at the second-stage bandwidth.

E.4 SCORE IS PAC

  • At second-stage bandwidth 4d, a refined Frobenius lower bound avoids the exponential-in-dimension constant of the earlier comparison.The earlier lower-bound constant decays exponentially with dimension, while the refined bound does not at this bandwidth.Source: §E.4 · pp. 31–32
  • The original-coordinate PAC bound combines approximation and estimation terms, with its sample-complexity specialization bounded by K εapp + ε.The theorem states a probability guarantee for the deployed prediction; the bound uses the approximation term plus finite-sample terms.Source: §E.4 · pp. 33–34
  • The proof separates correlation error, compared through Frobenius norms, from marginal error between predictions sharing the fitted correlation.The marginal contribution is bounded by coupling predictions that differ in the first-stage parameters.Source: §E.4 · p. 33

F TOY EXAMPLE

The toy example tests covariance estimation in a band-limited Gaussian reconstruction problem, where posterior rank and uncertainty vary with the band limit. The kernel score gives the best covariance estimates across band limits, while the log score has much larger gradients and struggles with low-rank structure.

F TOY EXAMPLE

  • Figure 6 pairs signal realizations with posterior covariance eigenvalues; at B=4 reconstruction is effectively deterministic, while larger B increases posterior uncertainty.For sufficiently large B, the limited observations cannot exactly reconstruct the signal.Source: Figure 6 · p. 34, F TOY EXAMPLE · p. 35
  • The task reconstructs a band-limited Gaussian signal from noisy subsampled observations, with prior rank min(2B+1,d).The prior is circulant, and observations select every m-th coordinate with Gaussian noise.Source: F TOY EXAMPLE · pp. 34–35
  • Across all band limits, the Gaussian kernel score gives the best Frobenius covariance estimate; Table 5 reports averages across five seeds.The table compares learned and true posterior covariance matrices by Frobenius distance.Source: Table 5 · p. 35, F TOY EXAMPLE · p. 35
  • With an added floor, the log-score gradient norm reaches 10^17, several orders larger than the kernel score's, and its low-rank estimates are noisy and incorrect.For large B, the log-score and kernel-score estimates are visually indistinguishable despite a remaining Frobenius-norm gap.Source: F TOY EXAMPLE · p. 35

G EXPERIMENT DETAILS

  • Across configurations, methods share the same backbone and vary the output head; results average five independent seeds.This setup is intended to make the methods directly comparable while accounting for run-to-run variability.Source: G EXPERIMENT DETAILS · p. 35

G.1 EVALUATION METRICS

The section defines common predictive scores and complementary calibration diagnostics for separating marginal, location, scale, and dependence errors. Calibration summaries are informative but not sufficient on their own.

G.1 EVALUATION METRICS

  • MSE measures mean accuracy, CRPS marginal fit, NLL, ES, and KS joint fit, while the variogram score targets dependence; lower values indicate better performance.MSE ignores predicted uncertainty, and the variogram score is shift-invariant.Source: §G.1 · pp. 35–37
  • Metrics use simulated predictive ensembles and unbiased estimators where needed, ensuring comparable evaluation of sample-based and distributional methods.The deterministic baseline is treated as a point mass, while the sample-based method uses its ensemble mean and covariance.Source: §G.1 · p. 36

G.2 CALIBRATION

  • Marginal PIT calibration checks only diagonal covariance information, so it cannot distinguish predictive dependence structures.Under calibration, pooled marginal PIT values are uniform on [0, 1].Source: §G.2 · p. 38
  • Rank histograms use pre-rank functions to diagnose location, scale, and dependence, with a diagonal ECDF consistent with calibration for that diagnostic.The diagnostics use simulated ensemble members exchangeable with the observation under the null.Source: §G.2 · p. 38
  • Mean, variance, and variogram diagnostics help localize bias, over- or underdispersion, and lag-specific dependence errors, but uniformity is necessary rather than sufficient.The variogram pre-rank is shift-invariant; the variance pre-rank is complementary to the mean-level diagnostic.Source: §G.2 · pp. 38–39

G.3 TASK DESCRIPTIONS

The experiments share task-specific trunks and probabilistic heads, with SCORE splitting marginal and core training. The tasks cover long-horizon series, gridded weather, depth maps, and station forecasts, including a graph-transform extension.

Shared experimental protocol

  • Across the four tasks, methods share a task-specific feature extractor but differ in probabilistic heads and losses; SCORE freezes its trunk and marginal head while fitting the core in stage two.All configurations use equal total epoch budgets and five seeds.Source: §G.3 · p. 39

Time series benchmark

  • The time-series benchmark uses PatchTST on eight datasets, including Electricity and Traffic outputs up to d = 165,504, with channel-independent predictive heads.The other tasks use ERA5 surface temperature on a 160 × 220 grid and NYU Depth v2 maps with 238,560 pixels.Source: §G.3 · pp. 39–40

Surface temperature prediction

  • ERA5 predicts three-day standardized temperature changes over a European grid using a U-Net trunk and a shared mean-and-marginal-variance head.SCORE and low-rank models add a second head for dependence.Source: §G.3 · pp. 39–40

Monocular depth estimation

  • The NYU Depth v2 task predicts cropped depth maps from RGB images, with training holes filled by nearest-neighbour interpolation and densely in-painted test maps.Its Depth Anything V2 trunk combines a frozen DINOv2 encoder and trainable DPT decoder.Source: §G.3 · p. 40

Graph-based temperature post-processing

  • EUPPBench predicts station temperatures from ensemble weather forecasts, using 115 stations after excluding locations with more than 50% missing values.The task evaluates reforecasts and operational forecasts and uses a graph neural network.Source: §G.3 · p. 40
  • The graph Fourier transform supplies a unitary spectral basis, but this graph-based covariance setup requires Cholesky decomposition for loss optimization and therefore offers no computational gain.Unlike the regular-grid setting, the graph is not vertex-transitive, so marginal variances require explicit normalization.Source: §G.3 · pp. 40–41

H.1 NUMERICAL RESULTS

SCORE performs strongly across time-series and ERA5 benchmarks, but its circulant assumptions are poorly suited to nonstationary depth maps. Runtime and calibration results show a favorable compute trade-off alongside persistent challenges in dependence calibration.

H.1 NUMERICAL RESULTS

  • SCORE ranks best on 20 of 24 time-series tasks (mean rank 2.2 versus 3.1 for kernel-score LorD) and beats diagonal and low-rank models on CRPS in 23 of 24 tasks.It also improves variogram score over diagonal and low-rank models on 22 and 21 tasks, respectively, while running near diagonal-model cost.Source: §H.1 · p. 41, Table 8 · p. 42
  • On ERA5, SCORE has the best or tied-best mean on every metric; its variogram score improves 7% and log score improves 1.3 nats per coordinate versus the NLL-trained diagonal model.The approximately stationary, distance-dependent temperature correlation is compatible with a circulant core.Source: §H.1 · p. 47, Table 16 · p. 47
  • On NYU Depth v2, SCORE ranks behind the diagonal model on every sample-based score; its CRPS is 0.0204 versus 0.0192 for the NLL-trained diagonal model.Depth correlations cluster at object boundaries rather than following a stationary pattern, and SCORE's log score explodes.Source: §H.1 · p. 47, Table 17 · p. 48
  • On EUPPBench, SCORE is competitive but ranks third overall, while the sample-based method ranks best and kernel-score LorD ranks second.SCORE produces more clustered, local dependencies than LorD, which estimates many correlations between distant stations.Source: §H.1 · p. 48, Table 18 · p. 48

H.2 RUNTIME ANALYSIS

  • Across datasets, SCORE generally delivers the largest performance improvement with the smallest computational increase among dependence-modeling baselines, and is the fastest such baseline.The compute-performance gains are not universal across metrics; the second stage improves performance even under decreasing epoch budgets.Source: §H.2 · p. 48, Table 19 · p. 50

H.3 CALIBRATION RESULTS

  • Almost all methods show good marginal calibration, but none is well calibrated for dependence, especially at short lags; SCORE is comparatively better at larger lags and on ETTh and EUPPBench.The authors note that no method truly recovers the dependence structure across these complex domains.Source: §H.3 · p. 49, Figure 11 · p. 51, Figure 12 · p. 52

H.4 ABLATION STUDIES

Ablations distinguish the contribution of the kernel score from covariance parameterization and test whether the staged core fit adds value. Kernel-score training generally helps low-rank models, while the second stage improves multivariate metrics across budgets.

Log score vs. kernel score

  • Kernel-score training beats NLL on MSE, CRPS, and ES with a majority in every dataset, significant after Holm correction; on VS, EUPPBench favors NLL.The comparison aggregates signs across data-set clusters rather than pooling task magnitudes.Source: §H.4 · p. 53

Loss and low-rank comparison

  • For LorD, the kernel score performs significantly better across metrics without an apparent benefit from increasing rank, while NLL's relative loss decreases with larger rank.The ablation tests ranks r ∈ {1, 2, 4, 8, 16}.Source: §H.4 · p. 53, Figure 13 · p. 53

Impact of stage 2 training and epoch budget

  • Stage-two training significantly improves multivariate metrics over first-stage-only training, even with very few epochs and poor marginal performance.The comparison uses univariate time series and EUPPBench and varies the total training budget.Source: §H.4 · pp. 53–54, Figure 14 · p. 54
Loading 2610.12096v1…