PaperlayerView exact revision on arXiv

Source-linked AI summary

Non-separable Dynamic Nearest-Neighbor Gaussian Process Models for Large spatio-temporal Data With an Application to Particulate Matter Analysis

Abhirup Datta, Sudipto Banerjee, Andrew O. Finley, Nicholas A. S. Hamm, Martijn Schaap

arXiv:1510.07130v3stat.AP

TL;DR

Continuous spatio-temporal Gaussian processes can provide fine-scale predictions and uncertainty, but large datasets make their computation difficult. This paper constructs DNNGPs using covariance-based neighbor selection to approximate these processes. The authors report more accurate predictions than prior analyses of the same data and identify long-term space-time structure relevant to LOTOS-EUROS performance.

  • Problem

    Large spatio-temporal covariance models require costly computation, while continuous-domain analyses have received limited attention and existing approaches may restrict inference or prediction.

    Source: §1 · pp. 2–3

  • Method

    DNNGPs approximate a spatio-temporal process by conditioning on covariance-defined nearest neighbors and are applied as a prior in a Bayesian analysis of European PM10 data.

    Source: §4.1 · p. 10, §7 · p. 20

  • Results

    The analysis yields more accurate predictions than previous studies of the same data and identifies long-term space-time structures that offer insight into LOTOS-EUROS performance.

    Source: §8 · p. 27

  • Takeaways & Limitations

    Long time-series analysis provides a perspective on CTM performance through residual space-time structures not captured by LOTOS-EUROS.

    Source: §8 · p. 27

    Reproducing observed CTM variability remains challenging, particularly during episodic conditions, and anthropogenic emissions are prescribed using static monthly, weekly, and hourly profiles.

    Source: §8 · p. 27

Abstract

from arXiv · show

Particulate matter (PM) is a class of malicious environmental pollutants known to be detrimental to human health. Regulatory efforts aimed at curbing PM levels in different countries often require high resolution space-time maps that can identify red-flag regions exceeding statutory concentration limits. Continuous spatio-temporal Gaussian Process (GP) models can deliver maps depicting predicted PM levels and quantify predictive uncertainty. However, GP based approaches are usually thwarted by computational challenges posed by large datasets. We construct a novel class of scalable Dynamic Nearest Neighbor Gaussian Process (DNNGP) models that can provide a sparse approximation to any spatio-temporal GP (e.g., with non-separable covariance structures). The DNNGP we develop here can be used as a sparsity-inducing prior for spatio-temporal random effects in any Bayesian hierarchical model to deliver full posterior inference. Storage and memory requirements for a DNNGP model are linear in the size of the dataset thereby delivering massive scalability without sacrificing inferential richness. Extensive numerical studies reveal that the DNNGP provides substantially superior approximations to the underlying process than low rank approximations. Finally, we use the DNNGP to analyze a massive air quality dataset to substantially improve predictions of PM levels across Europe in conjunction with the LOTOS-EUROS chemistry transport models (CTMs).

1. Introduction.

Fine-scale PM maps need calibrated predictive uncertainty, but continuous spatio-temporal process models are computationally difficult at large scale. The paper introduces DNNGPs as scalable continuous-domain processes for full Bayesian inference and prediction.

1. Introduction.

  • Dense covariance calculations typically require ∼n^3 flops and O(n^2) storage, making large spatio-temporal process models prohibitive.The relevant matrix is the n × n covariance matrix over observed space-time coordinates.Source: §1 · p. 2
  • Existing continuous space-time approaches often restrict inference to parameter estimation, rely on asymptotic uncertainty estimates, and use a different process model for prediction.The cited parameter-estimation approaches may be inappropriate for irregular observations, while prediction remains expensive for large n.Source: §1 · p. 3
  • DNNGPs approximate spatio-temporal processes through small neighbor sets, support non-separable covariance and full Bayesian inference, and require storage and memory linear in dataset size.They are defined as processes that support prediction at new space-time locations and are reported to approximate the underlying process better than low-rank approximations.Source: §1 · pp. 3–4
  • Sparse monitoring and statutory PM10 limits make fine-scale maps with quantified predictive uncertainty important for monitoring mitigation and compliance.The European Commission standards cited are 50 µg m−3 over 24 hours and 40 µg m−3 over a year.Source: §2 · p. 4

2. PM10 pollution analysis.

The study combines European PM10 monitoring data with LOTOS-EUROS output to produce continuous predictions, uncertainty estimates, forecasts, and estimates of residual space-time structure. Its two-year dataset uses rural background stations and daily CTM values matched to station locations.

2. PM10 pollution analysis.

  • CTM output is used as a covariate alongside station observations, with the analysis intended to yield realistic maps and dependence estimates that may inform transport-model improvement.The study combines observed PM10 with LOTOS-EUROS output and targets continuous maps with uncertainty, forecast maps, and residual space-time structure.Source: §2 · pp. 4–5

2.1. Study area.

  • The study domain covers 14 mainland European countries, with all data projected to the European Terrestrial Reference System 1989 Lambert Azimuthal Equal-Area coordinate system.The included countries are listed in the study-area passage.Source: §2.1 · pp. 5–6

2.2. Observed measurements.

  • Daily PM10 data span January 2008 through December 2009 at 308 rural background stations, with 41,761 of 224,840 potential observations missing and later predicted by the models.Stations above 800 m were excluded, and daily averages required at least 18 hourly measurements.Source: §2.2 · p. 6

2.3. LOTOS-EUROS CTM data.

  • LOTOS-EUROS v1.8 is a 3D lower-troposphere chemistry transport model, and its hourly PM10 output was averaged to daily values and matched to the station-coincident grid cells.The model uses a 25 km × 25 km grid and simulates particulate components separately; no attempt was made to match CTM and observation spatial support.Source: §2.3 · pp. 6–7

3. Scalable Dynamic Nearest-Neighbor Gaussian Processes.

The DNNGP construction starts with a continuous spatio-temporal Gaussian process and approximates its finite-dimensional distributions by conditioning on small neighbor sets. The resulting process has sparse precision and supports scalable inference and prediction across the domain.

3. Scalable Dynamic Nearest-Neighbor Gaussian Processes.

  • A spatio-temporal Gaussian process assigns multivariate normal distributions to finite sets of space-time locations, with covariance determined by C(·, · | θ).The domain is the product of a spatial region and a time domain.Source: §3 · p. 7
  • The construction defines a process from values on a reference set, using a positive-definite covariance matrix and a linear representation for locations outside that set.Choosing the parent covariance and kriging coefficients recovers the parent GP in the general framework.Source: §3 · p. 8
  • For each reference location, the approximation replaces conditioning on its full history with conditioning on a finite neighbor set, producing a multivariate normal approximation with sparse precision.The history contains earlier times and earlier locations at the same time in the specified ordering.Source: §3 · pp. 8–9
  • With neighbor sets limited to m points, the precision matrix has at most O(rm^2) nonzero elements, yielding a sparsity-inducing proper prior that approximates the parent GP.The columns of the factor matrix have at most m + 1 nonzero entries.Source: §3 · p. 9
  • At locations outside the reference set, the process uses only neighbor values in its conditional expectation and determines their coefficients from the corresponding covariance matrices.The coefficient vector has at most m nonzero entries when the neighbor set has m points.Source: §3 · p. 9
  • The DNNGP likelihood can be evaluated in O(rm^3) flops, while evaluating densities for finite sets outside the reference set is linear in their size.The computational savings rely on small neighbor sets and the diagonal residual covariance.Source: §3 · p. 10

4. Constructing Neighbor-Sets.

The paper defines spatio-temporal neighbors through covariance, because non-separable covariance may not admit a universal distance ordering. It develops a parameter-free simple selection and an adaptive selection that searches smaller eligible sets for the true neighbors.

4. Constructing Neighbor-Sets.

  • The method defines a point as nearer when its covariance with the target is higher, avoiding a universal distance function for non-separable covariances.The covariance-based neighbor ordering depends on the covariance function and its parameters.Source: §4.1 · p. 10

4.1. Simple Neighbor Selection.

  • Simple neighbor sets combine √m spatial neighbors with √m temporal neighbors from preceding time points, producing a parameter-independent set.For locations outside the reference set, the construction uses the Cartesian product of the spatial and temporal neighbors.Source: §4.1 · pp. 11–12
  • The simple set can miss true neighbors: it contains 7 of 9 for θ = 1 but only 3 of 9 for θ = 2, and extreme spatial- or temporal-only variation can leave m − √m points uncorrelated.The mismatch arises because true nearest neighbors vary with covariance parameters.Source: §4.1 · p. 11, §4.2 · p. 12

4.2. Adaptive Neighbor Selection.

  • Adaptive selection avoids recomputing correlations over the full history at every parameter update, which otherwise costs O(r^2 + nr) flops.Its eligible sets use spatial-temporal rectangles; a point is excluded when its rectangle contains more than m points.Source: §4.2 · pp. 12, 14
  • Adaptive selection searches eligible sets of approximately 4m points, which are guaranteed to contain the true m nearest neighbors for every covariance parameter choice.The sets are computed once before MCMC; searching within them reduces complexity from O(nr + r^2) to O(4m(n + r)).Source: §4.2 · pp. 13–14

5. Bayesian DNNGP model.

The Bayesian DNNGP model places a sparse spatio-temporal process prior on regression random effects, including for non-separable covariance structures. Gibbs and Metropolis updates support full posterior inference, while the model also provides prediction at unobserved locations and times.

5. Bayesian DNNGP model.

  • The model relates each response to predictors, a spatio-temporal random effect, and independent Gaussian noise, and permits missing observations.The random effect is constructed as a DNNGP from a parent GP with a non-separable covariance function.Source: §5 · p. 15

5.1. Gibbs’ sampler steps.

  • The hierarchical model assigns priors to the process parameters, noise variance, regression coefficients, and reference-set random effects, enabling full posterior inference.The sampler uses Gibbs and Metropolis steps.Source: §5 · pp. 15–16

5.2. Metropolis step.

  • Sequential random-effect updates and a random-walk Metropolis update for θ avoid expensive matrix decompositions, keeping the number of flops per update linear in the number of locations.Adaptive neighbor-set updates are handled using eligible sets.Source: §5.1 · p. 16, §5.2 · pp. 16–17

5.3. Prediction.

  • Posterior samples support prediction at unobserved reference locations and at locations outside the reference set.For outside locations, the model constructs a neighbor set and draws random-effect and response samples.Source: §5.3 · p. 17

6. Synthetic data analyses.

Synthetic data experiments compare full GP, simple and adaptive DNNGP, and Gaussian Predictive Process models across three space-time covariance patterns. DNNGP models achieve results comparable to the full GP with computational advantages, while the predictive-process model has worse fit and prediction in these experiments.

6. Synthetic data analyses.

  • The three simulated datasets represented short spatial and long temporal range, long ranges in both dimensions, and long spatial and short temporal range.The correlation surfaces and contours are shown in Figure 4.Source: §6 · p. 17, Figure 4 · p. 18
  • For each dataset, the study compared full GP, simple- and adaptive-neighbor DNNGP, and bias-corrected GPP by parameter estimates, observed-data fit, RMSE at 500 holdout locations, and 95% predictive-interval coverage.DNNGP models were fit with m = 16, 25, and 36; GPP used a regularly spaced 8 × 8 × 8 = 512-knot grid within the domain.Source: §6 · p. 18
  • DNNGP and full GP models provide comparable parameter estimates, fit, and holdout prediction, while DNNGP has substantial computational advantages; GPP has worse fit and prediction.For n = 3375, GPP required about twice the CPU time of DNNGP despite substantial dimension reduction.Source: Table 1 · p. 19, §6 · p. 20
  • The full GP recovered generating parameter values except τ² for Dataset 1, and both DNNGP variants likewise captured true mean and process parameters except τ² for Dataset 1.Simple and adaptive DNNGP estimates differed very little, and estimates showed negligible differences across m = 15, 25, and 36.Source: §6 · p. 20

7. Analysis of Airbase and LOTOS-EUROS CTM data.

The Adaptive model with 25 neighbors had the best fit, while the Simple model with 36 neighbors had the lowest RMSPE; DNNGP predictions also compared favorably with earlier spatial regression models. The analysis estimated PM10 dependence ranges and illustrated station-level predictions and maps, with temporal smoothing potentially obscuring highly dynamic events.

7. Analysis of Airbase and LOTOS-EUROS CTM data.

  • The estimated spatial and temporal ranges were about 45–60 km and 30–33 days; Simple DNNGP ranges were longer than Adaptive ranges.The reported spatial range was much shorter than Hamm et al. (2015)'s 500–1500 km, attributed to accounting for temporal correlation.Source: §7 · p. 21, Figure 5 · p. 23
  • Adaptive m=25 had the lowest DIC and D, while RMSPE decreased with more neighbors and was lowest for Simple m=36.All models achieved reasonable coverage rates.Source: Table 2 · p. 22, §7 · p. 23
  • Observed-station examples show CTM downward bias, improved fit with spatiotemporal effects over non-spatial regression, and wider intervals for missing observations.Figure 6 shows observed and fitted/predicted PM10 at three representative stations.Source: §7 · p. 23, Figure 6 · p. 24
  • DNNGP models had lower RMSPE and bias, with comparable R2, than the time-invariant SVI and SVC models in Hamm et al. (2015).The comparison used April 1–14, 2009 holdout metrics.Source: §7 · p. 23, Table 3 · p. 25
  • On a 10-core Xeon CPU, Adaptive m=25 required about 24 hours for 25,000 MCMC iterations, versus approximately 260 hours on one CPU.The update of random effects across locations can be distributed across CPUs.Source: §7 · p. 25
  • For two example dates, DNNGP maps broadly resembled Hamm et al. (2015)'s SVC maps, though April 3 pollution over western France and northern Spain appeared clearer in Hamm et al.The DNNGP smooths across days, which may improve predictive performance while capturing highly dynamic events less well than day-specific models.Source: §7 · p. 25, Figure 7 · p. 26

8. Conclusion.

The conclusion presents DNNGP as a scalable, flexible prior for full Bayesian space-time inference and reports improved PM prediction alongside long-term residual temporal structure not captured by LOTOS-EUROS.

8. Conclusion.

  • DNNGP storage and floating-point operations are linear in the number of measurements.Source: §8 · p. 27
  • Long-term European PM10 analysis finds residual temporal structure dominates spatial structure after accounting for LOTOS-EUROS, with the temporal range physically consistent with PM10 lifetime and synoptic conditions.The authors relate PM10 variability to alternating meteorological conditions that can persist for days to weeks.Source: §8 · p. 27
  • CTM reproduction of observed variability remains challenging for episodic conditions, while static month, weekday, and hour emission profiles may leave variability insufficiently detailed.The authors suggest meteorological effects may improve modeling and remove the monthly signature found in this analysis.Source: §8 · p. 27
  • DNNGP yields more accurate predictions than previous studies of the same data and provides a long-term space-time perspective on CTM performance.The authors distinguish this perspective from analyses of individual pollution days and associated CTM performance.Source: §8 · p. 27
  • DNNGP can serve as a proper sparsity-inducing prior in Bayesian hierarchical models for full inference at arbitrary spatio-temporal resolutions, including random effects for non-Gaussian responses.Full posterior distributions for the underlying process are available at arbitrary location and time points.Source: §8 · pp. 27–28

the space and time variations differ significantly.

The section reports that Dynamic NNGP outperforms LAGP across three simulated space-time range scenarios and describes nonstationary extensions alongside remaining validation needs.

the space and time variations differ significantly.

  • 45 to 60 km spatial and 30 to 33 days temporal effective ranges characterize the European PM10 residual dependence after CTM adjustment.LAGP is unsuitable for this analysis because it is not a proper Gaussian Process and lacks global space and time decay parameters.Source: the space and time variations differ… · p. 35
  • Dynamic NNGP performs better than LAGP in all three simulated range scenarios; LAGP performs worse when temporal range dominates, consistent with its isotropic covariance assumption.The comparison uses repeated simulations on a 15 × 15 × 15 space-time grid with RMSPE based on 500 holdout locations.Source: the space and time variations differ… · p. 35
  • For covariate-dependent nonstationary variance, the proposed extension models the stationary process v(s, t) with DNNGP because physical neighbors may not have the highest process correlations.The suggested conditioning sets can otherwise contain few points highly correlated with a given location.Source: the space and time variations differ… · p. 36
  • A smooth-process model for nonstationary variance can use DNNGP approximations for both v(s, t) and log σ2(s, t), but the latter has a non-Gaussian full conditional.Source: the space and time variations differ… · p. 36
  • O(n) flops suffice to evaluate the conditional likelihood across observations for Metropolis updates, retaining scalability.Source: the space and time variations differ… · p. 36
  • Nonstationary extensions still require studies of parameter-estimation and kriging accuracy, random-effect MCMC convergence without conjugate Gibbs updates, and more general covariance structures.These investigations are identified as future work outside the paper's scope.Source: the space and time variations differ… · p. 37
Loading 1510.07130v3…