Source-linked AI summary
Scalable Sparse Autoregressive Variational Inference for Non-Gaussian Spatial Fields
Paul F. V. Wiemann, Matthias Katzfuss
TL;DR
Non-Gaussian latent-field posteriors are analytically intractable, while existing approximations can be too slow or restrictive. The paper combines sparse NN-AR structure, normalizing-flow conditionals, and a shared neural conditioner; it reports polylogarithmic per-iteration complexity and O(1) amortized variational parameters.
Problem
Non-Gaussian likelihoods make latent-field posteriors analytically intractable, and existing approximations can be too slow or restrict posterior shape or dependence.
Source: §1 · p. 1Method
The framework mirrors sparse NN-AR structure in a variational family with normalizing-flow conditionals and a shared neural conditioner.
Source: §1 · p. 1, §3.1 · p. 3, §3.2 · p. 3Results
The framework achieves polylogarithmic per-iteration complexity and O(1) amortized variational parameters; preliminary GP-classification and Arctic ice-coverage results demonstrate expressiveness and scalability.
Source: §5 · p. 4Takeaways & Limitations
Preliminary results on GP classification and Arctic ice-coverage data demonstrate expressiveness and scalability for non-Gaussian latent spatial fields.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Latent field models are essential for spatial statistics, but the posterior distribution of the latent field, even with a Gaussian process (GP) prior, is analytically intractable whenever the likelihood is non-Gaussian. Existing approximations are either too slow or too restrictive for large non-Gaussian datasets. We present a scalable variational inference framework that mirrors the sparse nearest-neighbor autoregressive structure of the prior in the variational family, uses normalizing-flow conditionals for expressiveness, and amortizes all variational parameters via a single shared neural conditioner. Doubly stochastic optimization with reduced ancestor sets achieves polylogarithmic complexity per iteration.
1 Introduction
Latent-field posteriors are analytically intractable in these settings, making scalable approximate inference essential. Existing methods are too slow or restrict posterior dependence or shape; the proposed framework combines sparse structure, flexible conditionals, and amortization.
1 Introduction
- Latent-field posteriors are analytically intractable across applications including classification, count data, spatial point patterns, censored observations, and generative learning.Scalable approximate inference is therefore essential.Source: §1 · p. 1
- MCMC is prohibitively slow at large n, while Laplace, mean-field, and sparse-inverse-Cholesky approximations respectively restrict posterior shape or dependence.Laplace approximations can severely underestimate non-Gaussian uncertainty; sparse-inverse-Cholesky VI remains Gaussian.Source: §1 · p. 1
- The framework combines Vecchia-type linear-cost structure, normalizing-flow conditionals, and a shared neural conditioner to amortize parameters and support inference at scale.Source: §1 · p. 1
2 Sparse Autoregressive Framework
The framework uses a nearest-neighbor autoregressive prior and mirrors its sparse structure in the variational family. Reduced ancestor sets and doubly stochastic gradients yield polylogarithmic per-iteration complexity, with accuracy improving as ρ increases.
2.1 NN-AR prior
- The NN-AR prior replaces conditioning on all earlier field values with a nearest-neighbor subset C_i after maximin ordering.Because |C_i|=O(ρ^D) is constant in n, prior evaluation and simulation cost O(n).Source: §2.1 · p. 2
2.2 NN-AR variational family and scalable ELBO
- The variational family mirrors the NN-AR structure, and its ELBO decomposes into per-index terms depending on each node’s ancestor set A_i.Each term collects likelihood, prior, and variational-density contributions.Source: §2.2 · p. 2
- Reduced ancestor sets R_i have size O(ρ^D log n) and suffice for accurate gradient estimation; mini-batches and Monte Carlo samples yield O(ρ^D log n) complexity per iteration.The method uses doubly stochastic gradients; the reported KL accuracy improves monotonically with ρ.Source: §2.2 · p. 2, Figure 2 · p. 3
- Figure 1 marks selected indices, conditioning sets C_i, full ancestor sets A_i, and reduced sets R_i; for large i, |A_i|>|R_i|.Source: Figure 1 · p. 2
3 Expressive Amortized Variational Family
Normalizing-flow conditionals make the variational family expressive, while a shared neural conditioner amortizes parameters across locations. The architecture encodes neighbor features, pools them, and decodes flow parameters.
3.1 Normalizing flow conditionals
- Each conditional uses a sparse autoregressive normalizing flow, with a strictly monotone transformation and a conditioner mapping neighbor context to flow parameters.Under regularity conditions, ARNFs can be universal density approximators; for sufficiently large ρ, the family can approximate any posterior arbitrarily well.Source: §3.1 · p. 3
- The flow family contains mean-field VI and Gaussian SIC-VI as special cases.Mean-field corresponds to ρ=1 with an affine transformation; SIC-VI uses an affine conditional with linearly context-dependent means and context-independent scales.Source: §3.1 · p. 3
3.2 Amortized conditioner
- A single conditioner γ₀ is shared across locations, with location-specific context s_i such as coordinates, neighbor distances, or local observations.This replaces O(n) location-specific conditioner parameters with a shared parameterization.Source: §3.2 · p. 3
- An encoder-pooling-decoder architecture encodes neighbor features, aggregates them over C_i, and produces ψ_i; all variational parameters are updated in each mini-batch step.The total parameter count falls from O(n) to O(1).Source: §3.2 · p. 3
4 Results
The amortized approach produced accurate posterior maps for a larger Arctic ice-coverage dataset with a fixed parameter count, while matching non-amortized accuracy on a smaller subset. In GP binary classification, the two-piece-normal NN-AR family captured posterior skewness and achieved a higher ELBO than the Gaussian baseline.
GP classification
- In GP binary classification, the two-piece-normal NN-AR family captured posterior skewness and achieved a higher ELBO than the Gaussian baseline.Figure 3’s right panel compares estimated densities for f11.Source: §4 · p. 3, Figure 3 · p. 4
Arctic Ocean ice coverage
- At n = 21,855, amortized inference produced accurate posterior maps with 1,398 parameters; the roughly 200,000-parameter non-amortized version initially failed to converge reliably.The 1,398-parameter count is independent of n.Source: §4 · p. 4
- At n = 512, amortized inference visually matched non-amortized accuracy using 1,398 versus 4,580 parameters.The non-amortized count is approximately 9n.Source: §4 · p. 4
5 Conclusion
The paper presents a scalable variational inference framework for non-Gaussian latent spatial fields, combining sparse NN-AR structure, normalizing-flow conditionals, and neural amortization. Its preliminary GP-classification and Arctic ice-coverage results demonstrate expressiveness and scalability.
5 Conclusion
- The framework combines sparse NN-AR structure, normalizing-flow conditionals, and neural amortization for non-Gaussian latent spatial-field posteriors.Amortized inference reduces variational parameters to O(1), with polylogarithmic per-iteration complexity.Source: §5 · p. 4
- Preliminary GP-classification and Arctic ice-coverage results demonstrate the framework’s expressiveness and scalability.Source: §5 · p. 4