Source-linked AI summary
Cases for the nugget in modeling computer experiments
Robert B. Gramacy, Herbert K. H. Lee
TL;DR
The paper questions the default use of zero-nugget Gaussian-process interpolation for computer experiments because deterministic codes can still contain bias, assumption violations, and unpredictable computational behavior. It develops the case for estimating a nugget and reports better predictive coverage and statistical protection in several common settings. The supported scope includes sparse designs, nonstationarity, and problematic deterministic simulators, while relying on correlation and determinism assumptions that may fail.
Problem
Zero-nugget GP interpolation assumes deterministic computer experiments and omits measurement error, despite broader uncertainty from simulator bias, assumption violations, and computational behavior.
Method
The paper compares smoothing GP surrogates with estimated nuggets against zero-nugget interpolators across sparse designs, assumption violations, and deterministic-simulator scenarios.
Results
Estimated-nugget surrogates show better statistical properties, including improved predictive coverage, across the paper’s numerical examples.
Takeaways & Limitations
A nugget can protect surrogate fits against poor modeling assumptions and outputs whose deterministic computation does not reliably represent the intended function.
Abstract
from arXiv · showhide
Most surrogate models for computer experiments are interpolators, and the most common interpolator is a Gaussian process (GP) that deliberately omits a small-scale (measurement) error term called the nugget. The explanation is that computer experiments are, by definition, "deterministic", and so there is no measurement error. We think this is too narrow a focus for a computer experiment and a statistically inefficient way to model them. We show that estimating a (non-zero) nugget can lead to surrogate models with better statistical properties, such as predictive accuracy and coverage, in a variety of common situations.
1 Introduction
The paper challenges zero-nugget interpolation as the default for computer-experiment surrogates, arguing that estimated nuggets address conceptual as well as numerical problems. It reviews stationary GP models and their nugget extension, which smooths outputs rather than forcing exact interpolation.
- 1 Introduction: Zero-nugget interpolation is conventionally justified by treating computer experiments as deterministic, but the paper argues this definition is outdated and many modern experiments are stochastic.Examples include agent-based and finite-element simulations governed by stochastic rules.
- 1 Introduction: The paper focuses on deterministic simulators for which a nugget should nevertheless be used because interpolation can impose undesirable modeling consequences.The authors distinguish this conceptual argument from numerical-stability arguments and note additional concerns about smoothness and derivatives.
- 1 Introduction: Zero-nugget models can produce inferior predictive surfaces when designs are sparse or standard assumptions are violated, including assumptions explored through numerical examples.The paper also examines determinism as a source of inferior predictive behavior.
- 1.1 GP basics: A stationary Gaussian process models finite collections of simulator evaluations jointly as multivariate Gaussian and bases covariance on relative input positions.Its Gaussian correlation specification produces a smooth interpolating curve, with predictive variance zero at observed inputs and increasing with distance from them.
- 1.1 GP basics: Adding a nugget g introduces a Kronecker-delta covariance term, makes the model analogous to fitting noisy data, and leaves predictive variance gσ^2 at observed inputs.Unlike the zero-nugget model, the resulting GP no longer interpolates the data.
- 1.1 GP basics: The paper uses a Bayesian implementation, while stating that its arguments also apply under the frequentist paradigm.Parameters σ^2, d, and g can be inferred by maximizing the multivariate-normal likelihood or through Bayesian analysis.
2 Examining the model assumptions
The paper examines errors and assumption failures that make strict interpolation problematic even for nominally deterministic simulators. It argues that smoothing with a nugget can protect fits against simulator bias, nonstationarity, correlation misspecification, and unpredictable computation.
- 2 Examining the model assumptions: The zero-nugget focus treats measurement error as absent, although computer-model outputs can contain multiple other sources of uncertainty.The paper frames this as a broader modeling issue rather than merely a numerical-stability concern.
- Simulator bias: Simulator bias arises because computer simulators are mathematical approximations of the real world, and calibration with real data introduces noise equivalent to a nugget.The measurement-error likelihood formulation and nugget covariance formulation can be re-parameterizations of the same model.
- Simulator bias: When real data are unavailable, the simulator’s bias cannot be estimated, so smoothing its output is proposed instead of insisting that the surrogate interpolate it.The paper presents this as a reason to allow smoothing even when the simulator is treated as deterministic.
- The stationarity assumption: Limited data make fully nonstationary models difficult to fit, while a nugget can protect against moderate stationarity deviations that may be hard to detect.The paper reports that even minor stationarity violations can yield poor statistical properties without a nugget.
- Correlation assumptions: Misspecified or difficult-to-fit correlation functions provide another reason to allow smoothing rather than rely on an assumed correlation structure.The paper identifies the correlation specification as an approximation to reality and marks this as a scope limitation of the modeling setup.
- The assumption of a deterministic simulator: A theoretically deterministic simulator may behave unpredictably because of pseudo-randomness, chaotic behavior, numerical approximation, or convergence effects.The paper distinguishes nonmodelable determinism from theoretical but not numerical determinism.
- The assumption of a deterministic simulator: In the simulator example, 10% of runs can converge to a sub-optimal local solution, making zero-nugget interpolation sub-optimal and motivating random-process smoothing.The paper connects this setting to synthetic and real examples in later sections.
3 Statistically better fits with the nugget
Across sparse designs, violated stationarity or anisotropy, and technically deterministic iterative simulations, estimated-nugget GPs produce statistically better fits than no-nugget interpolators.
- 3.1 Protecting against misfits with sparse data: The nugget model had lower mean square error on average across 10,000 repeated 20-point uniform designs.The paired t-test comparing the models reported p < 2.2×10^-16.
- 3.1 Protecting against misfits with sparse data: With sparse data, nugget smoothing produced reasonable confidence bands, while no-nugget interpolation generated extreme predictions and unstable bands.The comparison used 20 available datapoints in a one-dimensional simulated example.
- 3.2 Poor coverage: Under nonstationarity, the no-nugget model could cover only 6.5% of the true function for one random design, whereas nugget-model median and mean coverages were near 90%.Across 100 repeated designs, three quarters of no-nugget trials under-covered by more than 10%.
- 3.2 Poor coverage: Similar no-nugget under-coverage appeared for the 2-d exponential function and the 5-d Friedman function with repeated uniform designs.The Friedman experiment examined the stronger effect of an isotropic correlation choice under anisotropy.
- 3.3 Challenging determinism in computer simulation: For an iterative deterministic simulator, estimated-nugget GP fits were sensible despite discontinuities and unpredictable behavior caused by convergence to different local minima.The simulator was deterministic technically, but its initialization-dependent optimization produced bias, discontinuities, and instability near x = 0.
- 3.3 Challenging determinism in computer simulation: When determinism was challenged as an assumption about the data-generating mechanism, smoothing with a nugget was clearly preferred to interpolation.The comparison used coverage and square-root Mahalanobis distances from 100 repeated uniform designs of size 20.
4 A modern computer experiment
The LGBB simulator models rocket forces across speed, angle of attack, and sideslip, but regime changes and nonconvergence challenge the practical determinism assumption. Cross-validation compares Gaussian-process coverage with and without an estimated nugget, including treed models for nonstationarity.
- LGBB simulator: The LGBB simulator computes six rocket-force outputs from speed, angle of attack, and sideslip using a 1.4-million-cell computational-fluid-dynamics mesh.Although theoretically deterministic, runs can fail to converge or produce inferior solutions for arbitrarily nearby inputs.
- Experimental setting: The experimental design concentrates finer grids around Mach one, where abrupt subsonic-to-supersonic regime changes make simulation difficult.The boundary region is especially challenging because stationarity assumptions may fail there.
- Evaluation: Coverage was evaluated by 20-fold inverse cross-validation, training on about 159 of 3041 pairs and predicting at roughly 3009 remaining locations, repeated 100 times.The resulting 2000 coverages per predictor are not uniform over the input area because the design is concentrated around Mach one.
- Gaussian process: An estimated nugget is much better behaved for the Gaussian-process coverage analysis, although it slightly over-covers.The authors attribute this mainly to design nonuniformity and prior choices under nonstationarity and nonconvergence.
- Treed Gaussian process: For treed Gaussian processes, estimating the nugget improves coverage by centering it more closely around 90%, whereas the no-nugget version tends to undercover.The authors judge the nugget especially important when nonstationary models address practically nonconvergent computer experiments.
5 Discussion
The discussion argues that a nugget is justified beyond numerical stability because computer-code outputs contain approximation and implementation artifacts. Smoothing can avoid interpolating those idiosyncrasies and may improve statistical prediction when simulator outputs are sparse or imperfect.
- Statistical rationale: The paper argues that a nugget protects against poor fits caused by violated modeling assumptions, not merely numerical instability.This extends earlier arguments for nuggets based on computational convenience to broader statistical concerns.
- Statistical rationale: The authors connect nugget use to bias–variance tradeoffs and shrinkage, which can improve estimators and predictors despite deterministic-experiment conventions favoring interpolation.They present interpolation as an application-specific feature that may conflict with standard statistical practice.
- Conclusion: The paper concludes from synthetic and real-data examples that smoothing computer-code outputs can be statistically preferable to forcing interpolation.Its discussion frames the recommendation as a supported preference rather than a universal computational requirement.
- Modeling workflow: The paper's idealized workflow distinguishes physical measurements, a mathematical model, computer-code implementation, and an emulator.Computer realizations are expensive, motivating an emulator to reduce computational cost and time.
- Modeling workflow: Interpolating computer-code outputs also interpolates implementation idiosyncrasies, whereas a nugget can smooth rough edges introduced by biased or erratic approximations.The authors acknowledge that this may not place predictions exactly at the mathematical model, but argue it offers a better chance of approaching the physical process.
- Alternative approach: The discussion notes that rougher interpolating correlations such as the Matérn can improve fit and matrix properties, but require an additional smoothing parameter that is difficult to infer.This is presented as an alternative to using a nugget, with complications beyond numerical behavior.