PaperlayerView exact revision on arXiv

Source-linked AI summary

Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes

Percy S. Zhai, Max Pachman, Peverel D. Shipley, Jungeum Kim

arXiv:2610.12850v1stat.MLcs.LGstat.COstat.ME

TL;DR

As observation size grows, SBI can struggle to recover increasingly concentrated posteriors. The paper jointly learns a structured summary and posterior through two-way guidance, and establishes guarantees for posterior accuracy and learned-summary sufficiency under finite simulation budgets.

  • Problem

    SBI must recover increasingly concentrated posteriors as observation size grows, while predetermined summaries can discard posterior information.

    Source: §1 · pp. 1–2

  • Method

    The approach jointly learns a summary and posterior, structuring posterior dependence on the summary so posterior sufficiency simplifies learning and the objective guides the summary toward sufficiency.

    Source: §1 · pp. 1–2

  • Results

    The paper establishes finite-simulation guarantees for posterior accuracy and learned-summary sufficiency, and gives simulation-budget conditions for recovering increasingly concentrated posteriors.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Structured summaries can counterintuitively improve posterior approximation, with performance depending on summary dimension, architecture, and training settings.

    Source: §5 · p. 12

    The guarantees assume bounded Lipschitz regularity, bounded parameter ranges, and uniform parameter sensitivity for finite-dimensional networks.

    Source: §3 · p. 9

Abstract

from arXiv · show

Bayesian simulation-based inference (SBI) can struggle to recover increasingly concentrated posteriors as observation size grows. We consider learning sufficient summary statistics jointly with the posterior as a general principle for scalable inference in various SBI methods. A common measure taken in SBI is dimension reduction through predetermined summaries, which may partially discard posterior information. Our approach instead imposes structure on how the posterior learner depends on a learned summary (e.g., affinity), and learns the summary simultaneously with the posterior. We identify an important mechanism named two-way guidance: under a structural restriction on the summary, the posterior sufficiency of the learned summary simplifies posterior learning, whereas a suitable posterior objective guides the learned summary towards posterior sufficiency. This mechanism explains a counterintuitive finding that the structural restriction on the summary statistics improves posterior learning, rather than limit its performance. We establish theoretical guarantees on both posterior estimation error and the learned summary's departure from sufficiency, thus identifying conditions on the simulation budget for recovering increasingly concentrated posteriors as observation size grows.

1 Introduction

As observation size grows, SBI must recover increasingly concentrated posteriors while correctly describing remaining uncertainty. This paper studies jointly learning structured summaries and posteriors, with two-way guidance linking posterior sufficiency to simpler posterior learning and summary learning.

1 Introduction

  • Jointly learned summaries organize posterior learning, while the posterior objective guides summaries toward retaining the information needed for inference.This two-way guidance is studied across multiple SBI methods.Source: §1 · pp. 1–2
  • Structured summary learning extends beyond quantile learning, addressing an understudied question of when simple dependence on a learned summary can make posterior inference more efficient.The motivation includes a contrast with predetermined summaries, which may discard posterior information, and with prior joint-learning approaches.Source: §1 · pp. 1–2
  • Structural restrictions can improve posterior learning because learned coordinates can preserve information while making the posterior-learning function simpler.The method need not recover a predetermined sufficient statistic; the benefit of saving simulation budget can outweigh summary-learning costs and structural error.Source: §1 · pp. 2–3
  • The paper establishes finite-simulation guarantees for posterior accuracy and learned-summary sufficiency under objectives whose excess loss controls posterior error.The bounds account for joint approximation, estimation, and optimization errors, with explicit conditions for approximate sufficiency.Source: §1 · p. 3
  • Sufficient simulation-budget conditions are given for recovering increasingly concentrated posteriors as observation size grows, accounting for summary-learning costs.The analysis compares conditioning on raw data with the proposed method.Source: §1 · pp. 1, 3

2 Structured Summary Learning

The paper learns a summary jointly with a posterior learner whose dependence on that summary is structured. Across three SBI methods, posterior objectives can guide the summary toward retaining posterior information, while the structure simplifies posterior learning.

2 Structured Summary Learning

  • The three approaches place the learned summary in different components: DGQB’s quantile potential, flow matching’s joint velocity, and SB-GAN’s conditional generator.Each method learns the summary jointly through an objective involving simulated parameters.Source: §2 · p. 3

2.1 Deep Generative Quantile Bayes

  • A sufficient summary with affine dependence lets DGQB learn functions of source noise alone; its objective also penalizes omitted posterior-relevant features and guides the summary toward sufficiency.Jointly learning the encoder and potential allows nonlinear features of X to enter through the summary while dependence on the learned summary remains affine.Source: §2.1 · p. 4

2.2 Conditional Flow Matching

  • In conditional flow matching, affine summary dependence simplifies velocity learning, while prediction errors backpropagated through the summary guide it to retain features relevant to parameter displacement.The summary-flow loss is detached from the encoder so encoder updates come from the parameter-displacement objective.Source: §2.2 · p. 6

2.3 Bayesian GAN

  • In SB-GAN, an affine generator uses learned summary coordinates, while a full-data critic detects posterior discrepancies and guides the encoder; a finite critic may miss relevant differences.The critic must retain the original observation: a summary-only critic could match a summary-conditioned posterior even when the summary has discarded posterior information.Source: §2.3 · p. 7

3 Posterior Accuracy and Learned Sufficiency

The analysis relates posterior accuracy to both summary insufficiency and fitting error, and studies how affinity changes the learning bound. These results retain costs and conditions that limit what the bounds establish.

Posterior error and summary insufficiency

  • Posterior error separates summary insufficiency from fitting the oracle posterior given the summary, and the reverse bound makes accurate posterior learning require retained posterior information.Thus, under calibration, improving the posterior learner alone cannot make an information-discarding summary approach the full-data optimum.Source: §3 · p. 8

The learning cost under affinity

  • Under affinity, the sufficient learning bound replaces growing functional dimension ndX + p with p, while retaining the cost of learning the summary and auxiliary networks.The bound is sufficient for the stated classes, not a lower bound for every raw-data architecture.Source: §3 · p. 9
  • For the implemented moving-summary SFM sampler, posterior calibration remains an additional requirement; regression convergence alone does not establish it.The stated bound also requires the listed regularity, loss-cover, reference-criterion, optimization, and objective-approximation conditions.Source: §3 · p. 9

4 Experiments

With simulation budgets and schedules held fixed, the experiments compare learned-summary and raw-data learners as observation size grows. Learned summaries improve posterior approximation in the Gaussian model and lower energy scores in Brock–Hommes, with method-specific differences.

4 Experiments

  • The experiments compare DGQB, SFM, and SB-GAN with raw-data counterparts as observation size increases, holding simulation budget and training schedule fixed within each pair.The Gaussian model supplies a closed-form posterior target; Brock–Hommes evaluates draws against a fixed truth using the energy score.Source: §4 · p. 9, §4.1 · p. 9, §4.2 · p. 11

4.1 Gaussian Conjugate Model

  • At n = 4096, summary DGQB, SFM, and SB-GAN mean distances are 0.012, 0.017, and 0.022 versus raw-data 0.170, 0.210, and 0.037; NPE and NRE also improve at larger sizes.The distances compare learned and exact posteriors; Figure 2(a) reports mean and standard deviation across three seeds.Source: §4.1 · pp. 10–11, Figure 2 · p. 11
  • Jointly learned summaries help posterior learners follow the shrinking exact Gaussian posterior, but SB-GAN remains overly concentrated and displaced from its center.The residual SB-GAN errors concern posterior spread and location.Source: §4.1 · pp. 10–11

4.2 Brock–Hommes Model

  • At n = 2000, summary DGQB, SFM, and SB-GAN mean energy scores are 0.032, 0.027, and 0.076 versus raw-data 0.143, 0.382, and 0.124; summaries score lower at every tested size.The reductions are particularly pronounced for DGQB and SFM.Source: Figure 3 · p. 12, §4.2 · p. 12

5 Discussion

The discussion frames joint summary and posterior learning as a scalable SBI principle, with two-way guidance explaining how structural restrictions can improve posterior approximation.

5 Discussion

  • Two-way guidance combines a structured summary that simplifies posterior learning with a posterior objective that steers the summary toward posterior sufficiency.The discussion notes that performance depends on summary dimension, architecture, and training settings, and identifies adaptive dimension selection as future work.Source: §5 · p. 12

A Methods and Implementation Details

The appendix describes how learned summaries enter five SBI methods and their training or sampling procedures. It then outlines theoretical results on posterior accuracy, summary sufficiency, and simulation requirements.

A Methods and Implementation Details

A.1 Deep Generative Quantile Bayes (DGQB)

  • DGQB learns a conditional vector quantile map from a convex potential; its affine potential representation jointly trains functional coefficients and a sufficient summary.The potential must remain convex in its source variable, and centered batch summaries simplify the objective while its gradients update the summary network.Source: §A.1 · p. 15

A.2 Summary flow matching (SFM)

  • SFM replaces the observation block of a joint flow with a learned summary, training the parameter velocity so its loss also updates the encoder.For posterior sampling, the summary ODE is solved backward once, then the joint flow is integrated forward for fresh parameter sources.Source: §A.2 · p. 16

A.3 Summary Bayesian GAN (SB-GAN)

  • SB-GAN jointly trains an affine-summary generator and encoder while its critic compares generated and simulated parameters at the same full observation.The critic gradient passes through the generator to the summary network; retaining the full observation in the critic is essential to the stated guidance interpretation.Source: §A.3 · p. 16

A.4 Neural posterior estimation (NPE)

  • NPE jointly trains a conditional density estimator on learned summaries by minimizing negative log-likelihood, using a masked autoregressive flow.At sampling, the observed summary is computed once and Gaussian base variables are mapped to parameter draws through the fitted conditional flow.Source: §A.4 · p. 17

A.5 Neural ratio estimation (NRE)

  • NRE jointly trains a summary encoder and classifier to distinguish dependent parameter–observation pairs from independent pairs.For a fixed summary, the optimal classifier odds equal the posterior-to-prior density ratio wherever that ratio is defined.Source: §A.5 · pp. 17–18

B General Theory for Posterior Accuracy and Scalability

  • The theory section establishes posterior-risk guarantees, conditions for recovering a sufficient summary, and simulation budgets for increasingly concentrated posteriors.It also quantifies the learning cost of affine summary dependence.Source: B General Theory for Posterior… · p. 18

B.1 Proof of Lemma 1

The proof compares the fitted posterior, the posterior given the learned summary, and the full-data posterior using Wasserstein bounds and conditional couplings. Averaging gives a posterior-risk guarantee without requiring a training objective or calibration condition.

B.1 Proof of Lemma 1

  • The triangle inequality splits fitted-versus-full posterior error into summary-induced information loss and mismatch between the fitted and summary posteriors.Squaring and averaging gives the first inequality in the proof.Source: §B.1 · p. 18
  • The proof constructs a coupling of the full posterior and fitted summary posterior, then uses conditional independence given the summary and auxiliary draw to relate it to the summary posterior.The conditional-mean decomposition supplies the corresponding squared-error identity.Source: §B.1 · pp. 18–19
  • Averaging over training yields Rf ≤ 2Rp without requiring a training objective or calibration condition.Source: §B.1 · p. 19

B.2 Proof of Theorem 2

Theorem 2 converts excess-loss control and calibration into posterior-risk control; consistency follows when approximation, statistical, and optimization errors meet the stated conditions.

B.2 Proof of Theorem 2

  • When ΔF = 0 and the statistical and optimization errors vanish, Rp and Rf converge to zero for fixed n and classes.The proof also gives Rf ≤ 2Rp.Source: §B.2 · p. 19

B.3 Sufficient conditions for statistical and optimization consistency

  • For fixed n, if ∆F = 0 and the calibration condition of Theorem 2 holds, then Rp → 0 and Rf → 0 as N → ∞.Source: §B.3 · p. 20
  • A finite uniform δ-cover and empirical minimization over representatives with δN ↓ 0 give ξN ≤ δN, so Theorem 2 yields both risk limits.The cover construction assumes an integrable loss envelope and finite admissible covers at every positive accuracy.Source: §B.3 · p. 20
  • For finite-sample or growing-n bounds, optimization error must be controlled at the displayed statistical rate; an unspecified vanishing rate suffices only for fixed-n consistency.For adversarial objectives, errors in the inner critic maximization must also be controlled.Source: §B.3 · p. 20

B.4.1 Function class complexity under affinity

The section compares loss-class complexity under affine summary dependence with raw-data dependence, accounting for the cost of learning the summary and auxiliary networks. It also bounds complexity for jointly trained finite-dimensional networks.

B.4.1 Function class complexity under affinity

  • Affinity reduces the function-approximation dimension from ndX + p to p while retaining the summary dimension and encoder learning cost.The comparison also accounts for auxiliary-network costs; the affine setup includes affine dependence on a learned summary.Source: §B.4.1 · pp. 21–22
  • The complexity comparison is an upper-bound result for the specified classes, since additional structure in a raw-data architecture can reduce its complexity.Source: §B.4.1 · p. 22

B.4.2 Finite-dimensional networks

  • The finite-dimensional network bound counts the whole jointly trained architecture, with observation and summary dimensions reflected in Wn and the uniform sensitivity bound.The bound applies to jointly trained density and classifier networks used by NPE and NRE.Source: §B.4.2 · pp. 22–23

B.4.3 Finite sample error

This section characterizes finite-sample errors in summary-based posterior learning, including finite-cover bounds and sufficient simulation budgets. It examines how posterior accuracy relates to information lost by the summary and fitting or transport errors, with analyses of DGQB and SFM.

B.4.3 Finite sample error

  • A finite δ-cover bounds uniform empirical error by combining approximation across candidates with statistical deviation over the representatives.Approximating each candidate in both empirical and population criteria adds 2δ; the concentration constant preserves each method’s sampling structure.Source: §B.4.3 · p. 23

B.5 Proof of Corollary 3

  • The explicit finite-sample rate uses exponent s = p for affine summary dependence and s = ndX + p for the raw class, with each class’s own An and Pn.Source: §B.4.3 · p. 24

B.6 Posterior-scale accuracy and simulation budgets

  • Sufficient budgets make posterior and summary risks negligible relative to n^-1/2 uncertainty, while a general-scale budget can be stated without N inside a logarithm.The guarantees are sufficient, not necessary for raw conditioning, and retain encoder, auxiliary-network, and calibration costs.Source: §B.6 · pp. 25–26

C Method-Specific Guarantees and Two-Way Guidance

  • Applying the general guarantee to five SBI methods requires its conditions to hold uniformly over each method’s candidate class.The section then examines how two-way guidance operates through the methods’ different objectives.Source: C Method-Specific Guarantees and… · p. 27

C.1 Deep Generative Quantile Bayes

  • DGQB learns a conditional vector quantile map with the conditional Monge–Kantorovich dual objective (Kim et al., 2025).The method-specific analysis connects objective guidance, posterior error, and finite-source approximation.Source: §C.1 · p. 27

C.1.1 Two-way guidance

  • A posterior-sufficient summary removes DGQB’s information-loss penalty, while an insufficient summary incurs a population penalty that better potential fitting cannot eliminate.The population objective splits into nonnegative information-loss and potential-fitting gaps; zero information loss is equivalent to posterior sufficiency.Source: §C.1.1 · pp. 27–28

C.1.2 Posterior accuracy

  • For DGQB, the full-space objective calibrates posterior error with Γ(z) = 2Lz; the unit-ball calibration adds a boundary-dependent square-root term.The full-space result assumes the total potential is convex and globally L-smooth; the unit-ball result accounts for source points near the boundary.Source: §C.1.2 · p. 28

C.1.3 Finite sample error

  • DGQB’s training replaces the conjugate supremum with a sampled-source maximum, whose approximation is controlled together with statistical error.The potential cover measures the complexity of the complete posterior learner, including its encoder.Source: §C.1.3 · p. 29
  • Claim C.4 states statistical and source-maximum error guarantees under its stated conditions.The source assumptions specify small-ball constants for volume-uniform and radius-uniform sources.Source: §C.1.3 · p. 30

The actual training criterion satisfies

  • Theorem 2’s actual training criterion controls encoder and potential learning errors plus sample-maximum error; affinity changes the functional exponent from 1/(ndX + d + 2) to 1/(d + 2).The N^2 source–observation combinations come from two independent samples of size N, so they are not N^2 independent observations.Source: The actual training criterion satisfies · p. 30
  • The posterior and insufficiency risks satisfy corresponding bounds under Theorem 2, which retain source-maximum error explicitly.Unbounded observation or parameter domains, a fixed number of source points per maximum, and changing minibatch normalization need additional discrepancy control.Source: The actual training criterion satisfies · p. 31

C.2.1 Two-way guidance

  • SFM’s two-way guidance makes an insufficient summary incur a positive parameter-regression penalty, while posterior sufficiency removes that information-loss gap.The population decomposition also includes moving-summary and fitting error, which may remain even when the summary is sufficient.Source: §C.2.1 · pp. 31–32
  • The information-loss term is zero exactly when the learned summary is posterior-sufficient: π(· | X) = π(· | f(X)) almost surely.The moving-summary and fitting term can also reflect source noise in the interpolated summary.Source: §C.2.1 · pp. 31–32

C.2 Summary flow matching

  • The parameter-displacement objective guides summary learning, while the block-triangular flow uses the learned summary for posterior sampling.The stop-gradient excludes encoder derivatives from the summary loss, leaving summary learning to the parameter-directed update.Source: §C.2 · p. 31, §C.2.1 · p. 32, §C.2.2 · p. 33

C.2.2 Posterior accuracy

  • Under correct joint transport, the learned-summary sampler’s posterior discrepancy equals the expected discrepancy between the summary-conditioned and full-data posteriors.A posterior-sufficient summary therefore yields the full-data posterior; regression convergence alone does not establish correct joint transport or calibrate the fitted sampler.Source: §C.2.2 · p. 33

C.2.3 Finite sample error

The finite-sample analysis quantifies how affinity reduces the regression class’s functional dimension, then develops the SB-GAN population guidance mechanism and conditions for converting its loss into posterior accuracy.

C.2.3 Finite sample error

  • For the actual regression objective, including learned inputs and targets, affinity changes the functional exponent from 1/(ndX + d + 3) to 1/(d + 3), while retaining encoder and summary-velocity costs.Source: §C.2.3 · p. 34
  • The regression bound covers the complete objective, including changes to learned inputs and targets, under the stated boundedness and class conditions.Unbounded inputs require separate tail or truncation control for the empirical rate.Source: §C.2.3 · pp. 33–35

C.3 Summary Bayesian GAN

  • SB-GAN learns an affine-summary generator with a Wasserstein minimax objective and uses a full-observation critic to guide summary learning.Source: §C.3 · p. 35, §C.3.1 · p. 35

C.3.1 Two-way guidance

  • In SB-GAN, affinity structures how the summary enters generation, while the full-observation critic penalizes posterior differences lost by the summary.An insufficient summary incurs positive loss even after generator optimization; zero loss is attainable when the summary is posterior sufficient and the generator can represent its posterior.Source: §C.3.1 · p. 35

C.3.2 Posterior accuracy

  • The ideal adversarial loss calibrates posterior error on bounded supports and, under a moment bound of order greater than two, on unbounded parameter spaces.Both calibrations vanish at zero loss.Source: §C.3.2 · p. 36

C.3.3 Finite sample error

The analysis bounds the complete neural-critic objective and explains how NPE’s likelihood objective separates summary information loss from density fitting error. It then relates that objective to posterior error under bounded-support or moment conditions.

C.3.3 Finite sample error

  • Affinity reduces the generator’s functional dimension from ndX + p to p, while critic learning cost remains because the critic receives the full observation.The finite-sample cover includes encoder, generator, and critic; posterior guarantees additionally require controlling approximation, gradient-penalty, optimization, and numerical errors.Source: §C.3.3 · pp. 37–38

C.4.1 Two-way guidance

  • NPE excess decomposes into posterior information lost by the summary and conditional-density fitting error; improving the density estimator alone cannot remove the first gap.Source: §C.4.1 · pp. 38–39

C.4.2 Posterior accuracy

  • The density-estimation objective controls full posterior error, including summary insufficiency, with calibration for bounded supports and moment-controlled unbounded laws.The unbounded-space result allows laws including Gaussian posteriors under the stated moment control.Source: §C.4.2 · pp. 39–40

C.4.3 Finite sample error

The NPE analysis bounds empirical objective error while counting encoder and density-network parameters together, then shows how NRE classification loss guides summaries and can be calibrated to posterior error.

C.4.3 Finite sample error

  • The NPE statistical term counts encoder and density-network parameters together, and for fixed observation size and class its statistical term tends to zero as N grows.The rate requires bounded loss and uniform sensitivity for the complete density architecture; finite parameter count alone is insufficient.Source: §C.4.3 · p. 41

C.5.1 Two-way guidance

  • NRE’s binary cross-entropy penalizes posterior information lost through the summary, while the summary supplies information for distinguishing dependent from independent pairs.A constant summary incurs a positive information-loss gap whenever the full posterior depends on X.Source: §C.5.1 · pp. 41–42

C.5.2 Posterior accuracy

  • The classification objective controls posterior error when fitted classifier probabilities stay bounded away from zero and one, with calibration also available under a moment condition.The bound’s constant controls the fitted classifier; it does not require an upper bound on the true Bayes classifier.Source: §C.5.2 · pp. 43–44

C.5.3 Finite sample error

The NRE analysis bounds classification and finite-pool sampling error, with posterior convergence as training pairs and pool size grow under stated conditions. The appendix also records training settings and Gaussian-experiment encoder and model choices.

C.5.3 Finite sample error

  • A fixed proposal pool leaves posterior approximation error even as the number of resampled draws increases; the pool itself must grow.The bound trades within-cell transport error against cell-mass error, so resolving finer parameter regions can require a larger pool.Source: §C.5.3 · p. 45
  • For fixed observation size and class, vanishing objective approximation and optimization errors imply posterior errors tend to zero as both training-pair count N and proposal-pool size K grow.The empirical bounds require uniform classifier bounds and uniform sensitivity; finite networks alone do not ensure them on unbounded inputs.Source: §C.5.3 · p. 46

D.1 Training Settings

  • Training used minibatches of 256 and 20,000 iterations; adversarial methods used learning rate 10^-4 and 15 critic updates per generator update.Source: §D.1 · p. 47

D.2 Gaussian Conjugate Experiment

  • The Gaussian experiment fixed the DeepSets summary dimension at q = 32 across observation sizes and summary-based methods.NPE used five masked autoregressive flow transformations, while NRE used a three-hidden-layer classifier.Source: §D.2 · p. 47

D.3 Brock–Hommes Experiment

In the Brock–Hommes experiment, NPE and NRE with jointly learned summaries achieve lower mean energy scores than their raw-data counterparts at every tested observation size. Posterior contours provide qualitative diagnostics, but do not establish calibrated uncertainty or full-posterior accuracy.

D.3 Brock–Hommes Experiment

  • The five summary learners share a q = 64 DeepSets–LSTM encoder; NPE generates direct posterior draws, while NRE samples the learned ratio times the prior.For NRE, numerical convergence and effective-sample-size checks screen the sampler but do not certify learned-posterior accuracy.Source: §D.3 · p. 47, Additional NPE and NRE results · p. 50
  • Some wider distributions, particularly Raw FM and Raw NRE, extend beyond the shared axes, so visible contour fragments do not enclose the stated probability masses.The display range is based on pooled draws from DGQB, SFM, SB-GAN, and Raw B-GAN.Source: Interpretation of posterior contours · p. 48
  • Contour changes vary by method, observation size, and marginal: SB-GAN remains displaced, DGQB can exclude the generating parameter, and NRE lacks monotone contraction.A marginal region excluding the generating parameter does not establish miscalibration; these plots are qualitative diagnostics, not evidence of full-posterior accuracy.Source: §D.3 · p. 47, Interpretation of posterior contours · p. 48
  • NPE and NRE with learned summaries achieve lower mean energy scores than their raw-data counterparts at every observation size considered.The comparison uses the same first observed series and averages across three training seeds; NRE shows substantial seed-to-seed variation at n = 500.Source: Figure 5 · p. 50, Additional NPE and NRE results · p. 50

Related papers

Loading 2610.12850v1…