PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalized Matheron Variational Implicit Processes

Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen

arXiv:2610.07938v1cs.LG

TL;DR

Implicit-process priors can be sampled as functions, but their function-space densities are typically unavailable for posterior inference. GMVIP replaces prior-path values at inducing inputs and uses a shared pathwise construction with a tractable coefficient-space KL. The method recovers the standard inducing-variable variational GP family for Gaussian priors and is broadly competitive across the reported tasks.

  • Problem

    Implicit-process priors provide sample-forward function draws, but their function-space densities are typically unavailable, making posterior inference in function space difficult.

    Source: §1 · p. 1

  • Method

    GMVIP draws a prior path and replaces its values at inducing inputs with a Matheron-style correction, while sharing the pathwise map between prior and posterior to obtain a coefficient-space KL.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    GMVIP recovers the standard inducing-variable variational GP family for Gaussian priors and is reported as competitive across regression and classification, with a qualitative forecasting comparison favoring its forecasts over VIP and FTIP.

    Source: §1 · p. 2, §5.3 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    For general implicit priors, the empirical Matheron construction preserves the original prior mean and covariance in the population limit, but not necessarily the full process law.

    Source: §1 · p. 2, §6 · p. 9

    For non-Gaussian or highly structured processes, inference is under a surrogate prior that may differ from the original implicit-process prior.

    Source: §6 · p. 9

Abstract

from arXiv · show

Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression, classification, and forecasting with simulator-defined and retrieval-conditioned empirical trajectory priors show that GMVIP is broadly competitive with existing methods.

1 INTRODUCTION

Function-space inference aligns the variational target with predictions, but implicit-process priors rarely have tractable function-space densities. GMVIP replaces prior-path values at inducing inputs and uses a shared pathwise construction to obtain coherent posterior samples with a coefficient-space KL.

1 INTRODUCTION

  • Implicit-process priors make function-space inference difficult because their induced densities are rarely tractable.Existing approaches use finite measurement sets, local approximations, or alternative discrepancies to address the process KL.Source: §1 · p. 1
  • GMVIP replaces prior-path values at inducing locations and propagates the correction while retaining the residual path away from those locations.The replacement values are µ + Da; the operator ΨZ can be estimated from empirical prior-sample covariance.Source: §1 · p. 2, Figure 1 · p. 2
  • The shared pathwise map makes the variational regularizer exactly KL(qϕ(a)∥N(0, IM)).The prior and posterior differ only in the distribution of the inducing coefficient.Source: §1 · p. 2
  • GMVIP introduces a coherent Matheron inducing-value replacement family, with exact GP/SVGP recovery and population-level preservation of the original prior mean and covariance for general implicit processes.The family requires prior sampling and likelihood evaluation.Source: §1 · p. 2

2 RELATED WORK

GMVIP addresses function-space inference and implicit-process priors by changing the pathwise surrogate construction, rather than only enriching the coefficient posterior. Its fresh prior paths retain functional variation beyond the finite span used by VIP and FTIP.

2 RELATED WORK

  • Function-space variational inference approximates posteriors over predictors, avoiding measurement-set density-ratio estimation and local linearizations through a shared pathwise map.Prior work uses finite measurement sets, finite-dimensional approximations, or Jacobians for tractable function-space inference.Source: §2 · p. 3
  • GMVIP complements FTIP by modifying the pathwise surrogate instead of enriching q(a) with an invertible transformation.VIP and FTIP draws are confined to the span of pre-sampled prior functions, whereas GMVIP draws a fresh prior path for each sample.Source: §2 · p. 3
  • GMVIP applies the pathwise inducing-value structure of Matheron sampling to priors that are only sampleable.Sparse variational GPs use inducing variables for tractable objectives with a KL over inducing variables.Source: §2 · p. 3
  • Its empirical covariance-gain correction shares linear-update algebra with statistical interpolation and ensemble Kalman methods, but GMVIP optimizes a variational law over replacement values.GMVIP regularizes that law with a coefficient-space KL.Source: §2 · p. 3

3 BACKGROUND: FUNCTION-SPACE VARIATIONAL INFERENCE AND IMPLICIT PROCESSES

Function-space variational inference approximates the posterior process directly, matching the variational object to predictors rather than parameterizations. Its process-level KL is rarely available, so existing methods rely on finite approximations or alternative discrepancies.

3 BACKGROUND: FUNCTION-SPACE VARIATIONAL INFERENCE AND IMPLICIT PROCESSES

  • The observed dataset is represented by inputs X and outputs Y, with each input xn ∈ X and output yn ∈ RC.Source: §3 · p. 3
  • Function-space variational inference aims to approximate the posterior process p(f | D) directly, rather than a posterior over parameters.The objective compares distributions over predictors and avoids some redundancies and symmetries of weight-space inference.Source: §3 · pp. 3–4
  • The process-level KL is rarely available, motivating finite measurement-set estimates, marginal approximations, Gaussian-measure discrepancies, local linearizations, and adversarial density-ratio estimation.These approaches introduce choices about measurement or inducing locations, density or ratio estimation, and coherence of posterior samples.Source: §3 · p. 4

4 METHODOLOGY

GMVIP constructs posterior paths by replacing prior values at inducing inputs and extending the correction with an operator estimated from prior samples. Its shared pathwise map yields a coefficient-space KL, while theoretical properties establish process coherence and GP recovery, with moment preservation for general priors.

4 METHODOLOGY

  • GMVIP replaces inducing values on a prior path and propagates the residual correction to arbitrary inputs, using an empirical covariance operator estimated from a prior bank.The prior-matched operator uses ˆK(·, Z) ˆK(Z, Z)−1 and depends on the quality of the prior-bank estimate.Source: §4 · p. 4
  • Inference changes only the law of the finite coefficient vector; the surrogate prior and posterior share the same pathwise map.The empirical operator and variational covariance may be joint across outputs or block-diagonal for channel-wise approximations.Source: §4 · pp. 4–5

4.1 THEORETICAL PROPERTIES

  • Reusing the same sampled path and coefficient across query inputs makes the induced finite-dimensional distributions consistent and defines valid stochastic processes.Source: §4.1 · p. 5
  • The shared map reduces the augmented KL exactly to KL(qϕ(a)∥p(a)), and the function-marginal KL is upper bounded by that quantity.The bound is tight when a is recoverable from f, for example when ΨZ(Z) = Ida and D is invertible.Source: §4.1 · p. 5
  • With the matching covariance operator, GMVIP recovers the full Gaussian-process prior and standard inducing-variable variational GP family.For general implicit priors, the matching operator preserves the prior mean and covariance, but not necessarily the full process law.Source: §4.1 · p. 5
  • Empirical-operator setup costs include prior-bank evaluation, covariance construction, inducing Cholesky factorization, and applying the inverse inducing covariance; each posterior sample then requires a prior draw and pathwise correction.The setup is amortized across posterior samples.Source: §4.1 · p. 6
  • The construction’s identities hold conditionally on Z, ΨZ, µ, and D; optimizing these quantities using observed targets makes the surrogate prior data-adaptive.Source: §4.1 · p. 6

5 EXPERIMENTS

Experiments span synthetic diagnostics, small and medium regression, large-scale regression, and forecasting with simulator-defined and retrieval-conditioned empirical priors. The results show strong but task-dependent performance, with different methods leading on different benchmarks and metrics.

5.1 SYNTHETIC FUNCTION-SPACE DIAGNOSTICS

  • The synthetic BNN regression diagnostic tests whether methods represent plausible uncertainty in regions with sparse observations and extrapolation.The methods produce predictive distributions on a shared input grid for the same fixed dataset.Source: §5.1 · p. 6

5.3 FORECASTING WITH IMPLICIT AND EMPIRICAL PRIORS

  • GMVIP preserved coupled prey–predator oscillations more consistently and, at S = 20, achieved the lowest mean RMSE, NLL, and CRPS; prior predictive coverage was closest to 0.90.VIP and FTIP forecasts developed increasing phase and amplitude errors over the held-out interval.Source: Table 2 · p. 7, §5.3 · pp. 7–8, Figure 2 · p. 7

5.2 SMALL AND MEDIUM REGRESSION BENCHMARKS

  • GMVIP was best or second-best on several small- and medium-regression dataset–metric combinations, with strong RMSE and CRPS results across six datasets.Calibration was mixed, and no single posterior family dominated uniformly; simpler or alternative approximations remained competitive on Power and Yacht.Source: §5.2 · p. 7

5.3 FORECASTING WITH IMPLICIT AND EMPIRICAL PRIORS

  • Forecasting uses either a fixed predator–prey ODE prior or a target-specific bank of 2,048 historical electricity trajectories retrieved using calendar compatibility and observed-context similarity.The retrieval-conditioned electricity prior uses the observed prefix for both retrieval and posterior fitting, making it a two-stage empirical-Bayes procedure.Source: §5.3 · pp. 7–8
  • Across 75 windows, Empirical Gaussian had the lowest mean RMSE (48.44 versus GMVIP's 49.06), while GMVIP led mean mixture NLL, CRPS, and CQM.Empirical Gaussian coverage was closest to the nominal 80% and 90% levels, with GMVIP second on both.Source: Table 3 · p. 8, §5.3 · p. 8

5.4 LARGE SCALE REGRESSION

  • On Airline, GMVIP had the best RMSE, tied for best rounded NLL, and second-best CRPS; FBNN led YearPredictionMSD on RMSE, NLL, and CRPS.VIP and FTIP had the lowest CQM on both datasets; GMVIP was second on Airline and tied for second on Year.Source: Table 4 · p. 9, §5.4 · p. 9

6 DISCUSSION AND CONCLUSION

GMVIP replaces inducing values along prior paths, with a coefficient-space KL; experiments report competitive results alongside limits from surrogate-prior fidelity, posterior-family choice, and computational cost.

6 DISCUSSION AND CONCLUSION

  • GMVIP replaces prior-path values at inducing locations through an empirical Matheron-style correction, while its shared prior and posterior map yields a finite-dimensional coefficient KL.Source: §6 · p. 9
  • In Lotka–Volterra, GMVIP has the lowest mean RMSE, CRPS, and mixture NLL among learned methods at S = 20, and retains the best reported metrics against larger VIP and FTIP bases.Results vary by target, and mean NLL is sensitive to one difficult high-amplitude trajectory.Source: §6 · p. 9
  • In electricity, GMVIP has the lowest mean NLL, CRPS, and CQM, while Empirical Gaussian has the lowest mean RMSE and coverage closest to nominal.Source: §6 · p. 9
  • For non-Gaussian or highly structured processes, the replacement-map surrogate prior may differ from the original, and poor prior banks or inducing locations can degrade performance.Empirical GMVIP also costs more than coefficient-only VIP-style methods; the reported experiments use Gaussian coefficient posteriors, which may be insufficient for strongly multimodal posteriors.Source: §6 · pp. 9–10
  • The authors used generative AI for methodology and experimental-design feedback, literature work, and text editing, while independently evaluating suggestions and checking references and experimental claims.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper directs readers to detailed protocols and supplementary analyses for reproducing its theoretical and empirical results, and to an appendix containing proofs and further properties of the construction.

REPRODUCIBILITY STATEMENT

  • Appendix B specifies GMVIP training and evaluation algorithms, while Appendix C details datasets, preprocessing, splits, architectures, optimization, seeds, sampling, inducing configurations, likelihoods, and metrics.Held-out evaluation protocols are specified for predictive results.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Additional analyses cover surrogate-prior fidelity, classification, full UCI results, ablations, and Lotka–Volterra basis-size sensitivity.Source: REPRODUCIBILITY STATEMENT · p. 10

A MATHEMATICAL DERIVATIONS

A.1 PROOFS OF THE MAIN RESULTS

The proofs establish KL identities and bounds for the shared pathwise construction, verify GP recovery and prior-moment preservation, and show process coherence and regularity under stated conditions.

A.1 PROOFS OF THE MAIN RESULTS

  • Using the same global prior draw and coefficients across query sets gives a projectively consistent family that defines a valid stochastic process.The proof uses row-consistent replacement operators.Source: §A.1 · p. 13
  • The augmented KL equals the coefficient-space KL, while the function-space KL is bounded above by it and reaches equality when the coefficient is recoverable from the function.The equality condition includes a cardinal replacement operator with invertible D.Source: §A.1 · p. 13
  • With the matching covariance operator and standard-normal coefficients, the construction recovers the GP prior and standard SVGP family.The transformed process has the GP mean and covariance; the affine whitening relation maps Gaussian inducing-variable posteriors to coefficient posteriors.Source: §A.1 · pp. 13–14
  • Under the stated finite-moment assumptions, the surrogate prior preserves the original process mean and covariance function.Source: §A.1 · p. 14

A.2 ADDITIONAL THEORETICAL PROPERTIES

  • The finite-dimensional KL distortion is bounded by the residual–inducing-value mutual information and vanishes when the surrogate inducing law matches and the residual is independent.The bound may be vacuous for singular or empirical implicit priors.Source: §A.2 · p. 14
  • GMVIP samples inherit continuity, differentiability, or higher-order smoothness when both the prior paths and replacement-operator columns have the corresponding regularity.Source: §A.2 · p. 15

B OPTIMIZATION AND EVALUATION PROCEDURES

The procedures estimate an empirical replacement operator from prior trajectories, train the inducing-coefficient posterior on observed data, then draw coherent paths for held-out evaluation.

B OPTIMIZATION AND EVALUATION PROCEDURES

  • The empirical operator uses a prior trajectory bank to estimate the mean and covariance, with positive-definite empirical inducing covariance assumed.When prior parameters or inducing locations are optimized, the bank is reevaluated and empirical moments and the replacement operator are recomputed.Source: B OPTIMIZATION AND EVALUATION PROCEDURES · p. 15
  • The posterior path uses u(a) = µ + Da and adds the operator-weighted difference between inducing values and the prior path’s inducing evaluations.Vector outputs retain joint cross-output covariance; for electricity, inducing inputs are the 96 observed locations and exclude the held-out interval.Source: B OPTIMIZATION AND EVALUATION PROCEDURES · p. 15
  • Predictive-density evaluation uses an equal-weight Gaussian mixture for sample-based NLL, while CRPS and coverage use empirical posterior samples and central predictive intervals.Electricity uses a predefined nine-level quadrature for its coverage-related metric.Source: B OPTIMIZATION AND EVALUATION PROCEDURES · pp. 15–16
  • Training draws minibatches, coefficient samples, and coherent prior paths at the minibatch inputs and inducing locations, then updates the coefficient posterior and likelihood parameters.The algorithm returns the fitted coefficient posterior and likelihood parameters.Source: B OPTIMIZATION AND EVALUATION PROCEDURES · p. 16
  • For each evaluation iteration, GMVIP draws a coefficient sample and one coherent prior path at test inputs and inducing locations, forms posterior samples, then computes RMSE, NLL, CRPS, CQM, and coverage.Electricity metrics are computed independently for each of 75 held-out windows and summarized by the mean and sample standard errors. Lotka–Volterra metrics are evaluated on the held-out test interval and summarized by the mean and standard error across 20 target trajectories.Source: B OPTIMIZATION AND EVALUATION PROCEDURES · pp. 16–17
  • Figure 4 places methods on the same input grid: MAP gives a deterministic fit, whereas Bayesian and function-space methods show posterior uncertainty through predictive samples or bands.The synthetic regression experiment probes plausible uncertainty with sparse observations and extrapolation.Source: Figure 4 · p. 17

C.1 SYNTHETIC REGRESSION EXPERIMENT

The experiments specify synthetic and UCI regression settings and extend implicit-prior inference to sparse Lotka–Volterra forecasting, where methods use simulator samples rather than explicit parameter posteriors.

C.1 SYNTHETIC REGRESSION EXPERIMENT

  • The synthetic regression experiment adapts a dataset with 400 scalar observations and compares GMVIP with methods including VIP on a BNN-induced prior.The BNN has two tanh hidden layers of width (10, 10), and predictions are evaluated on a dense one-dimensional grid.Source: §C.1 · p. 17

C.2 UCI REGRESSION EXPERIMENTS

  • The UCI study evaluates nine scalar regression datasets using five seeds, random 90/10 splits, and held-out RMSE, NLL, CRPS, and CQM.All reported metrics are transformed back to the original target scale.Source: §C.2 · p. 18

C.3 LOTKA–VOLTERRA IMPLICIT-PRIOR EXPERIMENT

  • The prior is the simulator-induced distribution p(f), used directly as a black-box function sampler; simulator parameters are not optimized, unlike fitted parametric-generator inference.Methods use 80 noisy training observations at times t ≤ 15, leave 15 < t ≤ 20 unused, and forecast clean trajectory values on t ∈ (20, 30].Source: §C.3 · p. 19
  • GMVIP models dependence across prey and predator values over time with a full 192 × 192 inducing covariance and variational coefficient covariance.The 192 coefficients arise from 96 shared inducing times and two state variables.Source: §C.3 · p. 20

C.4 ELECTRICITY-LOAD EMPIRICAL-PRIOR FORECASTING EXPERIMENT

The experiments test GMVIP with a retrieved historical electricity-load prior, alongside large-regression and LeNet classification settings. In electricity forecasting, GMVIP has the strongest typical point and distributional performance, with overlapping confidence intervals against the strongest baselines.

C.4 ELECTRICITY-LOAD EMPIRICAL-PRIOR FORECASTING EXPERIMENT

  • The electricity prior is a retrieval-conditioned empirical process of historical trajectories, without a parametric demand model or optimized customer-specific calendar variables.The forecast uses the first 24 hours as observations and the next 24 hours as the held-out region.Source: §C.4 · pp. 21–22
  • GMVIP provides the strongest typical point and distributional forecasting performance, although its confidence intervals overlap those of the strongest baselines.Electricity-load results are medians across forecasting windows because RMSE varies substantially; calibration depends on the metric and nominal coverage level.Source: §C.4 · p. 23, Table 5 · p. 23

C.5 LARGE REGRESSION EXPERIMENTS

  • The large-regression experiments evaluate two scalar datasets using RMSE, NLL, CRPS, and CQM on held-out test splits.Year and Airline use fixed splits, with five seeds per dataset and method variant.Source: §C.5 · pp. 23–24

C.6 LENET CLASSIFICATION EXPERIMENTS

  • The LeNet classification setting uses deterministic convolutional features and a Bayesian classifier head, with a block-diagonal variational covariance across ten logits.This avoids the full 1000 × 1000 coefficient covariance of a joint formulation; cross-logit covariance is not explicitly parameterized.Source: §C.6 · pp. 24–25

D.1 FINITE-DIMENSIONAL SURROGATE-PRIOR FIDELITY

This experiment compares finite-dimensional GMVIP and VIP surrogate priors against the original BNN prior and its finite-sample reference floor. GMVIP is closer on the reported distributional and moment measures, while the study bounds its claims to one prior and one finite grid.

D.1 FINITE-DIMENSIONAL SURROGATE-PRIOR FIDELITY

  • At matched coefficient dimensions, GMVIP reduces SW2 by 77.4% at dimension 8 to 27.9% at 256, and W1 reductions range from 80.5% to 33.5%.GMVIP with M = 8 has lower joint and marginal distances than VIP with S = 256.Source: §D.1 · p. 27, Table 7 · p. 28
  • GMVIP mean error is 0.040 versus 0.034 for the true split and 0.195 for VIP; relative covariance error is 0.073 versus 0.056 and 0.369, respectively.These finite-bank measurements support the population second-order preservation result while quantifying finite-sample estimation error.Source: §D.1 · p. 27
  • Energy distance is 0.0056 for GMVIP, 0.0317 for VIP, and 0.0032 for the true split; MMD2 RBF is 0.0022, 0.0117, and 0.0011, respectively.The agreement across marginal, projected-joint, moment, energy, and kernel measures makes the conclusion insensitive to any single discrepancy measure.Source: §D.1 · p. 27
  • With M = 256, increasing the operator bank to B = 2048 yields SW2 = 0.066 and W1 = 0.043, near true-split floors of 0.064 and 0.045.The smaller-bank discrepancies are not strictly monotone in this five-seed experiment.Source: §D.1 · p. 27
  • SW2 is 0.072 versus the 0.064 true-split floor, and average marginal W1 is 0.049 versus 0.045; GMVIP reduces VIP distances by 71.4% and 76.9%, respectively.The maximum pointwise discrepancy is 0.069 versus 0.066 for the split and 0.312 for VIP; Figure 6 shows GMVIP tracking the split baseline across the interval.Source: §D.1 · p. 26, Table 6 · p. 27, Figure 6 · p. 28
  • The measurements cover one non-Gaussian BNN prior on one 301-point grid and do not establish infinite-dimensional process equality or uniform behavior across implicit priors.A distance near the true-split floor is not proof of equality.Source: §D.1 · p. 28

D.2 CLASSIFICATION EXPERIMENTS

Across the reported classification settings, GMVIP performs strongly, with its standing varying by dataset and representation. It is strongest overall in frozen-CLIP CIFAR-10 and on DynaSent, while the LeNet results favor it particularly on FashionMNIST.

D.2 CLASSIFICATION EXPERIMENTS

  • In LeNet classification, GMVIP has the best FashionMNIST error and Brier score and ties FBNN for best rounded NLL; on CIFAR-10 it remains among the best methods.SIP leads the CIFAR-10 LeNet comparison on NLL, error, and Brier score, while MFVI has the lowest ECE.Source: §D.2 · pp. 28–29
  • On frozen-CLIP CIFAR-10, GMVIP obtains 94.45% accuracy, NLL 0.1667, and ECE 0.0058, with best accuracy and NLL and second-best ECE.VIP has 94.42% accuracy and NLL 0.1700; SIP has the lowest ECE at 0.0057.Source: §D.2 · p. 29
  • At matched coefficient dimensions, GMVIP stays close to the true-split floor, while VIP needs a larger basis to reduce discrepancy.Figure 7 plots joint sliced W2 and average marginal W1 against coefficient dimension.Source: Figure 7 · p. 29, §D.2 · p. 30
  • On DynaSent Round 1, GMVIP attains 59.48% accuracy, NLL 0.8769, and AURC 0.2503; under Round-2 shift, it has the best NLL and AURC.Under Round-2 shift, GMVIP has 57.41% accuracy, behind VIP at 57.96%.Source: §D.2 · p. 30
  • Across FashionMNIST LeNet, frozen-CLIP CIFAR-10, and DynaSent, GMVIP is strongest on FashionMNIST, strongest overall on CIFAR-10 under the common fixed-learning-rate protocol, and strongest overall on DynaSent.The paper reports that classifier performance depends on representation and data regime.Source: §D.2 · pp. 28–30

D.3 FULL UCI REGRESSION RESULTS

  • Table 12 completes Table 1 with CRPS and CQM metrics for the experiments.Both are reported metrics in the full UCI regression results.Source: §D.3 · p. 30

D.4 ABLATION STUDIES

The ablations show that GMVIP’s residual path and an adapted prior matter for predictive behavior, while inducing counts and prior-bank sizes involve accuracy–calibration trade-offs. In the Lotka–Volterra basis-size analysis, GMVIP remains strongest across the reported learned-model metrics; the efficiency passage describes diagnostics but gives no comparative outcome.

D.4 ABLATION STUDIES

  • At B = 2048, joint and marginal prior-discrepancy error bars overlap the true-prior split floor at the resolution of the diagnostics.The smaller operator-bank trend is non-monotone.Source: Figure 10 · p. 31
  • M = 128 wins on eight datasets for RMSE and CRPS and seven for NLL, while M = 16 wins CQM on five datasets.Smaller M settings generally underfit relative to larger sets, so increasing M trades calibration against predictive accuracy rather than dominating uniformly.Source: §D.4 · p. 31, Table 14 · p. 34
  • B = 256 gives the most consistent predictive performance among tested prior-bank sizes, while smaller banks sometimes improve CQM but have poorer predictive metrics.Increasing to B = 512 does not improve consistently across datasets.Source: §D.4 · p. 31, Table 15 · p. 34
  • Inducing-only GMVIP overfits Boston, Concrete, Wine Red, and Yacht; on Boston, test RMSE/NLL are 4.09/2083.56 versus 3.32/3.11 for full GMVIP.On Concrete, inducing-only has lower training RMSE (1.66 versus 2.75) but higher test RMSE (5.39 versus 4.24).Source: §D.4 · p. 32, Table 16 · p. 35
  • Removing the residual path raises coefficient-space KL on Boston, Wine Red, and Yacht and can worsen test NLL by several orders of magnitude.The residual path supplies predictive uncertainty and acts as a structural regularizer, though it is not uniformly beneficial.Source: §D.4 · p. 33, Table 16 · p. 35
  • Tunable-prior GMVIP beats its frozen counterpart on every dataset and reported metric; Yacht RMSE rises from 0.32 to 4.11 when the prior is frozen.On Concrete, RMSE rises from 4.24 to 7.34, and on Energy from 0.51 to 2.80.Source: §D.4 · p. 34, Table 17 · p. 35

D.5 VIP AND FTIP BASIS-SIZE SENSITIVITY ON LOTKA–VOLTERRA

  • GMVIP has the lowest mean RMSE, NLL, CRPS, and ODE residual among learned configurations, with coverage closest to 0.90.This remains true as VIP and FTIP basis sizes increase; S counts global sampled-function basis elements, whereas GMVIP uses inducing values, so the analysis is not a direct equality of representation dimensions.Source: Table 18 · p. 36, §D.5 · p. 36

D.6 EFFICIENCY AND CAPACITY DIAGNOSTICS

  • The Airline efficiency diagnostic reports setup and training time, evaluation time, peak CUDA memory, latent or coefficient dimension, and prior-sample count.It contextualizes computational capacity and overhead; the passage does not report which method is more efficient.Source: §D.6 · p. 36, Table 19 · p. 36
Loading 2610.07938v1…