PaperlayerView exact revision on arXiv

Source-linked AI summary

Structure-Preserving Dimension Reduction and Surrogates for Partially Linear Bayesian Inverse Problems

Ziheng Zhang, Dingcheng Luo, Thomas O'Leary-Roseberry, Umberto Villa, Omar Ghattas

arXiv:2610.30306v1math.NAcs.NA

TL;DR

Mixed linear–nonlinear inverse problems pose a challenge because generic surrogates may not preserve known source scaling. SPPL combines blockwise reduction with an affine-in-source surrogate and derives approximation and error-control results. The paper reports orders-of-magnitude acceleration of online posterior sampling, while its PDE validation is limited to a synthetic two-dimensional coseismic problem.

  • Problem

    Generic nonlinear surrogates must learn the known linear source action from training data and do not enforce its source-scaling relation.

    Source: §1 · p. 2

  • Method

    SPPL constructs separate reduced spaces for outputs, sources, and media, represents the reduced map as affine in retained mean-centered source coordinates, and derives blockwise approximation and error-control criteria.

    Source: §1 · p. 2

  • Results

    The conclusion reports orders-of-magnitude acceleration of online posterior sampling and robustness to changes in source amplitude.

    Source: §6 · pp. 21–22

  • Takeaways & Limitations

    For Gaussian source priors, the optimal affine-in-source approximation coincides with the optimal blockwise ridge approximation.

    Source: §1 · p. 2

    The framework assumes a known partially linear decomposition and independent medium and source priors, while PDE validation is limited to a synthetic two-dimensional coseismic problem.

    Source: §6 · p. 22

Abstract

from arXiv · show

Many Bayesian inverse problems are characterized by two parameter vectors with distinct roles. A medium property, operator coefficient, or geometry block enters the forward model nonlinearly, whereas a volumetric, initial condition, or boundary condition source enters linearly. General-purpose nonlinear surrogates must relearn this known source dependence from data and may not preserve the correct source scaling outside the training regime. We introduce a structure-preserving partially linear (SPPL) framework that constructs separate reduced spaces for outputs, (linearly appearing) sources, and (nonlinearly appearing) medium variables and represents the reduced forward map as an affine function of the retained source coordinates, with coefficients that depend on the medium variables. Here, "affine" refers to the retained, mean-centered source coordinates: the physical forward map remains linear in the source, while the affine offset is induced by the source-prior mean and vanishes when that mean is zero. The analysis first identifies the optimal blockwise ridge approximation and then derives the optimal affine-in-source approximation under independent medium and source priors. A direct decomposition of the forward error into output, source, medium, and surrogate contributions yields blockwise spectral criteria and prior-predictive expected-KL control. For a Gaussian source prior, the optimal affine-in-source approximation coincides with the optimal blockwise ridge approximation. In a coseismic numerical example, a three-model controlled ablation on common block-reduced coordinates separates the effects of coefficient-matrix supervision and structure-preserving affine-in-source parameterization. Reduced-space surrogate MCMC provides orders-of-magnitude online acceleration relative to high-fidelity posterior sampling.

1. Introduction.

The introduction motivates blockwise treatment of inverse problems whose forward maps are nonlinear in medium variables but linear in source variables. It proposes SPPL to preserve that structure while reducing outputs, sources, and media, and outlines its theory and numerical validation.

1. Introduction.

  • Generic projected nonlinear surrogates do not enforce the source-scaling relation F(m, cs) = cF(m, s), motivating a structure-preserving representation.The introduction notes that such surrogates must infer the known linear source action from training data.Source: §1 · p. 2
  • SPPL builds separate reduced spaces for outputs, sources, and media, with a reduced forward map affine in mean-centered source coordinates and medium-dependent coefficients.The affine offset arises from the source-prior mean and vanishes for a zero-mean source prior.Source: §1 · p. 2
  • The analysis characterizes optimal blockwise ridge and affine-in-source approximations, decomposes forward error, and derives blockwise spectral criteria and prior-predictive expected-KL control.For Gaussian sources, the optimal affine-in-source and blockwise ridge approximations coincide.Source: §1 · p. 2
  • The numerical validation includes quadratic benchmarks and a PDE-based coseismic inversion, with a three-model ablation separating source parameterization from training supervision.The evaluation considers posterior fidelity, source-scale robustness, and reduced-space MCMC performance.Source: §1 · p. 2
  • SPPL preserves linear source action while learning medium dependence, derives separate output, source, and medium reduction criteria, and imposes affine source dependence in the surrogate class.The introduction positions it alongside mixed linear–nonlinear optimization, derivative-informed reduction, and surrogate modeling; unlike NEMO, it treats both parameter blocks as uncertain.Source: §1 · p. 3

2. Partially Linear Bayesian Inverse Problems.

This section sets out a Bayesian inverse problem with noisy observations and a partially linear forward map, then uses its block structure to motivate separate reductions and posterior-error control.

2. Partially Linear Bayesian Inverse Problems.

  • The Bayesian formulation models observations as a forward response plus Gaussian noise, with the unknown parameter assigned a prior.The section uses the posterior to quantify uncertainty in the unknown parameter.Source: §2 · p. 3

2.1. Whitened Bayesian formulation and forward-error control.

  • Prior-weighted forward accuracy controls prior-predictive posterior error, providing forward-error control under data generated from the prior and observation noise.The bound is stated for the prior-predictive law.Source: §2.1 · p. 4

2.2. Partially linear model and priors.

  • The partially linear map is nonlinear in the medium and linear in the source; the term describes the parameter-to-observable structure, not the differential operator.With Gaussian priors, the section specifies positive-definite medium and source covariance matrices and proceeds to separate block reductions.Source: §2.2 · p. 4

3. Structure-Preserving Reduction Theory.

The theory reduces outputs, sources, and nonlinear medium variables in separate blocks while preserving the source’s linear action. It characterizes optimal retained-coordinate approximations, decomposes their errors, and derives spectral criteria and expected-KL control under stated assumptions.

3.1. Basic setting and reduced coordinates.

  • The setup assumes independent medium and source priors and uses prior-informed metrics for reduced coordinates, with the Gaussian medium prior precision as the canonical medium metric.The source prior is assumed to have finite second moment and positive-definite covariance.Source: §3.1 · p. 5

3.2. Optimal blockwise ridge approximation.

  • The blockwise ridge class restricts maps to retained medium and source variables with outputs in the retained output space, and the section characterizes its L2 optimum.The conditional means of the forward operator and source given their retained coordinates are defined for fixed projectors.Source: §3.2 · p. 6

3.3. Practical affine-in-source class.

  • For Gaussian source priors, the optimal affine-in-source approximation adds no error relative to the optimal blockwise ridge approximation.The retained source coordinates are mean-centered, so the affine offset reflects the source-prior mean.Source: §3.3 · pp. 6–7

3.4. Direct affine error decomposition and exact residuals.

  • The affine approximation error separates into output, source, medium, and surrogate contributions; output and source residuals have exact spectral trace forms, unlike the conditional medium residual.The medium residual requires a Poincaré bound before an analogous spectral criterion is available.Source: §3.4 · p. 7, §3.3 · p. 7

3.5. Spectral reduction criteria and medium Poincaré control.

  • Output POD modes minimize discarded output energy, and the retained-source generalized eigenspace minimizes source residual after the output space is fixed.The medium reduction criterion uses a subspace Poincaré inequality; the stated bound requires a suitable prior and metric.Source: §3.5 · pp. 8–9

4. SPPL Construction, Offline Algorithm, and Reduced-Space Inference.

SPPL reduces output, source, and medium spaces separately, then models the reduced map through medium-dependent coefficients and an affine source action. Its coefficient regression averages over discarded medium directions, separating that reduction error from trainable surrogate error.

4. SPPL Construction, Offline Algorithm, and Reduced-Space Inference.

  • The construction maps each forward-error decomposition term to a reduced-space selection criterion and construction step.Table 1 summarizes this theory-to-algorithm correspondence.Source: Table 1 · p. 9

4.1. SPPL architecture and coefficient-matrix targets.

  • SPPL's network takes retained medium coordinates as input and predicts coefficients for an affine map of retained, mean-centered source coordinates.The source action enters analytically outside the network; only its medium-dependent coefficients are learned.Source: §4.1 · pp. 9–10
  • The coefficient targets are projected responses to the source-prior mean and to retained source-basis directions.Because the network sees only retained medium coordinates, the population-optimal predictors under squared coefficient loss are conditional expectations given those coordinates, not pointwise matches for every full medium realization.Source: §4.1 · p. 10

4.2. Coefficient loss and population identity.

  • The coefficient-loss identity separates unavoidable medium-reduction error from trainable surrogate error by averaging over discarded medium directions.The resulting medium-only coefficient targets define the population regression target.Source: §4.2 · p. 10

For the SPPL surrogate

The SPPL surrogate is trained through medium-only coefficient regression, with PDE computations organized to reuse an operator factorization. For Gaussian priors, posterior inference can then operate on retained coordinates using the reduced coefficient model and analytic source action.

For the SPPL surrogate

  • Minimizing the coefficient loss is equivalent to minimizing the SPPL surrogate error because the medium residual is independent of the network parameters.The identity links the computable training objective to the surrogate error.Source: For the SPPL surrogate · p. 10

4.3. Empirical coefficient training.

  • For Gaussian sources, the unweighted Frobenius coefficient loss is the empirical training objective, and the optimal affine-in-source approximation coincides with the optimal blockwise ridge approximation.The stated justification uses E[α] = 0 and E[αα^T] = I_rs.Source: §4.3 · p. 11

4.4. PDE-operator factorization reuse for the medium diagnostic and coefficient tar-

  • A single factorization of L(m) supplies the forward and adjoint solves needed for both medium-space construction and coefficient-target values.When the source-prior mean is zero, the mean-source solve is omitted; the solve count then reduces to rs forward and ry adjoint solves.Source: §4.4 · pp. 11–12

4.5. Reduced-space approximate posterior and inference for Gaussian priors.

  • For Gaussian priors, posterior inference is performed in retained medium and source coordinates, with discarded components reconstructed from their conditional Gaussian prior when needed.Online MCMC evaluates only the reduced coefficient model and analytic affine source action.Source: §4.5 · p. 12

5. Numerical Experiments.

The experiments compare blockwise and joint reductions on quadratic benchmarks, then use a controlled three-model ablation to test surrogate supervision and source parameterization in coseismic inversion. Across these studies, blockwise reduction remains competitive, structure-preserving surrogates improve data efficiency and source-scale robustness, and reduced-space MCMC accelerates online sampling.

5.1.1. Gaussian source prior: exactness of the affine restriction.

  • Block DF-LIS closely matches Joint DF-LIS across all three Gaussian-source regimes, while both prior-mean GNH methods have consistently larger error.This indicates little loss from block-separable reduction relative to joint DF-LIS in these benchmarks, while prior-averaged derivative information matters for basis quality.Source: §5.1.1 · p. 14, Figure 1 · p. 14, §5.1 · p. 14

5.1.2. Gaussian-mixture source prior: direct affine error and spectral control.

  • Under the Gaussian-mixture source prior, Block DF-LIS remains competitive with Joint DF-LIS, and the computable source-plus-medium bound tracks the exact contribution closely.The joint methods use the exact conditional expectation, while blockwise methods use the optimal affine-in-source approximation.Source: §5.1.2 · p. 14, §5.1.1 · p. 14, Figure 2 · p. 15

5.2.2. Three-model ablation: separating source supervision from surrogate parame-

  • The controlled ablation compares DIPNet, Matrix-loss SPPL, and Probe-loss SPPL on common block-reduced coordinates and matched offline PDE budgets.Probe-loss SPPL versus DIPNet isolates affine parameterization; Matrix-loss versus Probe-loss SPPL isolates direct coefficient-matrix supervision.Source: §5.2 · p. 14, §5.2.2 · pp. 16–17

5.2.1. Physical model and discretization.

  • The coseismic model represents medium-dependent shear modulus and linearly entering fault slip in a 500 × 250 km domain with 20 two-component GPS stations.The discretization has state dimension 145083, medium dimension 3101, and slip dimension 29; the priors are independent and zero-mean Gaussian.Source: §5.2.1 · pp. 15–16

5.2.3. Numerical results.

  • Both SPPL variants are substantially more data-efficient than DIPNet at smaller training budgets, with the gap narrowing as training data increase.Matrix-loss and Probe-loss SPPL are nearly indistinguishable; because Probe-loss SPPL uses the same realized probe–output pairs as DIPNet, results suggest the gain arises primarily from affine source parameterization.Source: §5.2.3 · pp. 18–19, Figure 4 · p. 19
  • Matrix-loss SPPL remains accurate across tested source-amplitude scales, whereas scale-specific DIPNets deteriorate away from their training amplitudes.The Combined DIPNet achieves comparable robustness after training on all three source-amplitude regimes.Source: §5.2.3 · p. 19, Figure 4 · p. 19
  • Across source radii and all four training budgets, both SPPL variants maintain low forward error, while DIPNet accuracy remains tied to training amplitudes.All models use identical retained spaces, so the reported differences reflect surrogate parameterization rather than dimension reduction.Source: §5.2.3 · p. 19, Figure 5 · p. 20
  • At N_m = 6400, online MCMC is at least 10^3 times faster than high-fidelity sampling, while the Matrix-loss SPPL posterior closely matches the high-fidelity posterior.Forward evaluation drops from approximately 2.5 s to below 5×10^-4 s, and the LA-pCN step from approximately 4 s to below 2×10^-3 s.Source: §5.2.3 · p. 20, Figure 6 · p. 21

6. Conclusion.

The conclusion presents SPPL as a framework for partially linear Bayesian inverse problems, with blockwise reduction, affine-in-source approximation, and prior-predictive error control. Its numerical validation demonstrates competitive reduction, improved source-amplitude robustness, and faster online posterior sampling, while remaining bounded by stated assumptions and a synthetic two-dimensional test case.

6. Conclusion.

  • For Gaussian source priors, the optimal affine-in-source approximation coincides with the optimal blockwise ridge approximation.The conclusion also reports competitive blockwise reduction for non-Gaussian source priors, and identifies structure preservation as the principal source of surrogate accuracy and data-efficiency improvement in the coseismic ablation.Source: §6 · p. 21
  • The framework assumes a known partially linear decomposition and independent medium and source priors, and its PDE validation is limited to a synthetic two-dimensional coseismic problem.The conclusion identifies extension to large-scale three-dimensional geophysical inversion as future work.Source: §6 · p. 22

Appendix A. Auxiliary projection and source-moment identities.

  • Appendix A collects projection and source-moment identities, including the centered source and its covariance relation.The listed identity states E[s − s̄] = 0 and Cov(s) = R^-1.Source: Appendix A · p. 22

Appendix B. Proof of the optimal blockwise ridge approximation.

The proof obtains the optimal blockwise ridge approximation by conditioning on the retained medium and source coordinates. Independence of the prior blocks lets the conditional expectation factor into separate medium and source terms.

Appendix B. Proof of the optimal blockwise ridge approximation.

  • The blockwise ridge approximation is the conditional expectation of the projected forward map given the retained medium and source coordinates.The proof identifies this conditional expectation as the L2 projection onto the space of retained-coordinate-measurable functions.Source: Appendix B · p. 22
  • Independence of the prior blocks and separate retained coordinates make the conditional law factorize across medium and source.The conditional expectation of G(m)s therefore factors into the conditional expectation of G(m) times that of s.Source: Appendix B · p. 23
  • Substituting the retained coordinates and the definitions of G* and s* yields the stated blockwise approximation.The proof then concludes that this establishes the result.Source: Appendix B · p. 23

Appendix C. Proof of the optimal affine-in-source approximation.

The proof constructs the optimal approximation within the affine-in-source class through conditional affine normal equations. It then uses inclusion of that class in the blockwise ridge space to show the affine optimum is also obtained by projecting the ridge approximation.

Appendix C. Proof of the optimal affine-in-source approximation.

  • The affine class is closed because its coefficient representation is an isometry, so its L2 projection exists.The proof represents its elements as a(m_r) + B(m_r)α.Source: Appendix C · p. 23
  • Independence of the retained medium and centered source coordinates, together with identity covariance of α, gives nondegenerate conditional affine normal equations and a unique minimizer.The proof uses E[α | m_r] = 0 and covariance I for α.Source: Appendix C · p. 23
  • Because the affine class is contained in the blockwise ridge class, projecting the ridge optimum onto the affine class gives the same result as direct affine projection.This follows from the nested closed-subspace projection property.Source: Appendix C · p. 23

Appendix D. Proof of exactness of the affine restriction for Gaussian source priors.

For Gaussian source priors, the optimal affine-in-source and blockwise ridge approximations coincide. The appendices also establish an orthogonal four-part forward-error decomposition and use it to obtain medium-error and prior-predictive expected-KL bounds.

Appendix D. Proof of exactness of the affine restriction for Gaussian source priors.

  • For Gaussian sources, the retained conditional source mean is the prior mean plus the projected mean-centered source, making the optimal affine and blockwise ridge approximations coincide.The proof uses independence of the retained and discarded Gaussian components under the source-prior covariance metric.Source: Appendix D · pp. 23–24

Appendix E. Proof of the direct affine error decomposition.

  • The direct affine error decomposition separates output, source, medium, and surrogate residuals, which are pairwise orthogonal under the stated prior assumptions.The output residual is pointwise orthogonal to the other terms, while independence of medium and source blocks supports the remaining orthogonality.Source: Appendix E · p. 24

Appendix F. Proof of the affine medium trace bound and expected-KL control.

  • The affine medium trace bound is averaged over the source prior and combined with the error decomposition to yield prior-predictive expected-KL control.The proof applies a subspace Poincaré inequality for almost every fixed source coordinate, then averages using prior independence.Source: Appendix F · pp. 24–25

Appendix G. Proof of the expected coefficient-loss identity.

The coefficient-loss identity decomposes expected coefficient loss into surrogate and medium approximation contributions. The appendices give benchmark-construction details and specify the coseismic elasticity model used in the numerical example.

Appendix G. Proof of the expected coefficient-loss identity.

  • The expected coefficient loss equals the affine surrogate error plus the affine medium contribution, with the medium residual identified using shared retained coordinates.The proof also establishes equivalence between the medium residual expressions used in the identity and the earlier criterion.Source: Appendix G · p. 25

Appendix H. Quadratic benchmark construction and analytic formulas.

  • The quadratic benchmark uses coefficient matrices generated from common random output and source rotations, with symmetric quadratic coefficients and gradual spectral decay.The appendix states that the rank-matching rule and benchmark dimensions are defined in the main text.Source: Appendix H · pp. 25–26, §H.1 · p. 26

H.1. Coefficient generation and parameter regimes.

  • The benchmark evaluates source-dominated, balanced, and medium-dominated regimes for Gaussian and Gaussian-mixture sources, using 4×10^4 and 2×10^4 joint samples, respectively.Blockwise affine errors and the medium bound are evaluated analytically.Source: §H.1 · p. 26

H.2. Exact unrestricted joint benchmark for the Gaussian-mixture source prior.

  • For Gaussian-mixture sources, the unrestricted joint benchmark introduces a latent mixture component and uses Gaussian conditioning to obtain conditional moments.The conditional mixture weights and componentwise conditional forward mean are then used to define the benchmark.Source: §H.2 · p. 27

I.1. Physical model and discretization.

  • The coseismic example uses a two-dimensional weakly symmetric Hellinger–Reissner elasticity model with tangential fault slip as the linearly entering source.The discretized state equation is K(m)x = Ss, so the forward map is nonlinear in the medium and linear in slip; the model has 3,101 medium and 29 source coefficients.Source: §I.1 · p. 28

I.2. Priors, reduced spaces, and synthetic data.

The coseismic study uses Gaussian priors and shared block-reduced spaces to compare three surrogate variants under matched solve budgets. It also evaluates source-scale robustness and posterior fidelity, while distinguishing prior-predictive theory from fixed-data diagnostics.

I.2. Priors, reduced spaces, and synthetic data.

  • The medium and slip priors use BiLaplacian precisions with Robin boundary terms, and the Gaussian medium precision also serves as its canonical Poincaré metric.The nominal correlation lengths are 11.2 and 89.4 for the medium directions and 30 for slip.Source: §I.2 · p. 28
  • The shared reduced-space construction uses 200 medium-prior samples and reuses PDE-operator factorizations; its cost is common to all three surrogate comparisons and included in reported offline cost.The affine medium diagnostic uses 12 forward and 17 adjoint solves per sample.Source: §I.2 · pp. 28–29

J.1. Matched-cost training data.

  • All three training variants use the same medium realizations and the same number of PDE right-hand-side solves per medium realization.Probe-loss SPPL and DIPNet share the realized probe–output pairs, while Matrix-loss SPPL uses coefficient-matrix targets.Source: §J.1 · p. 29

J.2. Probe-loss SPPL and its relation to Matrix-loss SPPL.

  • In the zero-mean setting, population Probe-loss and Matrix-loss SPPL have the same minimizer, so empirical differences reflect finite source-probe sampling.Both SPPL variants use the same coefficient network in this setting.Source: §J.2 · p. 30

J.3. Network architecture, optimization, and replication.

  • SPPL preserves zero output at zero source and exact scaling with retained source coordinates, whereas training and reporting use three paired seeds without best-seed selection.The study also specifies matched network depth and width, optimization, and the across-seed mean with a minimum–maximum envelope for Figure 5.Source: §J.3 · p. 30

J.4. Source-scale DIPNet variants.

  • DIPNet source-scale diagnostics train separate models at ctrain = 1/10, 1, and 10 for Nm = 100, 400, 1600, and 6400, reusing the same medium realizations across scales at each Nm.The ctrain = 1 model is the nominal DIPNet used in the three-model ablation; the other two are auxiliary scale-transfer variants.Source: §J.4 · p. 31

K.1. Retained-coordinate sampling, posterior accuracy, and online efficiency.

  • Posterior comparisons estimate fixed-data forward KL using high-fidelity posterior samples, while the theoretical bounds concern prior-predictive expected posterior KL.Online efficiency is assessed through forward-evaluation cost, wall-clock cost per MCMC step, and integrated autocorrelation time.Source: §K.1 · p. 31

K.2. Source-scale robustness.

The source-scale posterior diagnostic uses reference samples from the high-fidelity posterior for each test datum. It compares scale-specific DIPNets, SPPL, and a combined-data DIPNet at the largest training budget.

K.2. Source-scale robustness.

  • Each Figure 4(b) point evaluates the surrogate potential at samples from the high-fidelity posterior corresponding to that test datum.The surrogate is not used to generate the reference samples.Source: §K.2 · p. 32
  • The posterior diagnostic compares Matrix-loss SPPL, three separately trained DIPNets, and a Combined DIPNet across every test scale using the Nm = 6400 models.The Combined DIPNet is trained on the union of the three scale-specific datasets and therefore uses three times the probe data.Source: §K.2 · p. 32
  • Figure 5 broadens the source-scale evaluation to all four training budgets and includes the three scale-specific DIPNets alongside two SPPL references.The supplied passages specify the comparison scope but do not report its outcomes.Source: §K.2 · p. 32

Related papers

Loading 2610.30306v1…