PaperlayerView exact revision on arXiv

Source-linked AI summary

Resolution-Aware Experimental Design under Partial Identifiability

Sofianos Panagiotis Fotias

arXiv:2609.03686v1cs.LG

TL;DR

Persistent nuisance uncertainty can make the same observation support different structural meanings, so information gain alone need not identify which exclusions are reliable. RAED selects experiments by valid candidate-set resolution, and its theory and empirical studies show both when that objective differs from information gain and when it respects a genuine composite informativeness ordering.

  • Problem

    Expected information gain is standard when the objective is posterior concentration, but persistent nuisance uncertainty can make the same observation carry different structural meanings.

    Source: §1 · p. 1, §4 · p. 8

  • Method

    RAED selects the experiment with the smallest expected nonempty structural candidate set subject to false-exclusion control, with learned score-based rules calibrated for nuisance-average and positive-tail validity.

    Source: §1 · p. 3, §9 · p. 29

  • Results

    Under constrained sensing, RAED and information-based criteria select different physical experiments in Watt and especially WCA, while fluvial tail protection changes the selected experiment and replaces many hard-region false exclusions with ambiguity.

    Source: §8 · p. 28, §9 · p. 29

  • Takeaways & Limitations

    RAED makes explicit the trade-off between reliable structural exclusions across nuisance uncertainty and the amount of unresolved ambiguity that is acceptable.

    Source: §9 · p. 29

    Watt and WCA nuisance populations are too small to support the stronger positive-tail population certificates, so their Tail-risk quantities are empirical generalization diagnostics.

    Source: §8 · p. 28

Abstract

from arXiv · show

Experimental design is commonly framed as choosing the experiment expected to provide the most information. Under partial identifiability however, persistent nuisance uncertainty can make the same observation carry different structural meanings. We introduce Resolution-Aware Experimental Design (RAED), which selects an experiment by the smallest expected nonempty structural candidate set achievable subject to false-exclusion control. We prove an exact cross-nuisance aliasing separation: an experiment can be preferred by structural and full-latent information gain, average classification, and nuisance-marginalized informativeness while having arbitrarily poorer valid structural resolution. RAED nevertheless preserves the expected ordering under a genuine composite Blackwell comparison. To make this criterion operational, we develop a learned score-based implementation with finite-sample nuisance-average and positive-tail calibration, and characterize a rare-tail sample-complexity obstruction. Under constrained sensing, two subsurface-flow benchmarks exhibit genuine RAED--expected-information-gain (EIG) experiment-selection disagreements, with the clearest and largest held-out resolution differences in WCA. In a fluvial benchmark, tail protection changes the selected physical experiment and replaces hard-region false exclusions primarily with explicit ambiguity. In a mechanistic methane-oxidation benchmark, a prospectively specified 5\% false-exclusion tolerance also yields a nontrivial finite-sample population guarantee for tail-sensitive nuisance risk, with 95\% joint confidence across all three structural families.

1 Introduction

Expected information gain is natural when the goal is posterior concentration, but persistent nuisance uncertainty can make an observation support incompatible structural meanings. RAED instead chooses the experiment with the smallest expected nonempty candidate set under false-exclusion control.

1 Introduction

  • Persistent nuisance uncertainty can make the same observation carry different structural meanings, so a single nuisance-blind rule may be unable to safely assign it a precise structural conclusion.This is cross-nuisance structural aliasing, an experiment-level form of partial identifiability.Source: §1 · p. 2
  • In the four-family example, each row of Table 1 identifies the structural family at fixed nuisance state, yet the same observation is compatible with every family across nuisance states.A single nuisance-blind terminal rule therefore cannot assign that observation a unique structural meaning.Source: Table 1 · p. 2, §1 · p. 2
  • RAED minimizes expected nonempty structural candidate-set size subject to a declared false-exclusion tolerance, with validity ranging from pointwise to nuisance-average or tail-sensitive protection.The nuisance-tail level and false-exclusion tolerance set the trade-off between structural resolution and validity protection.Source: §1 · pp. 2–3
  • The paper shows that cross-nuisance aliasing can make information-based criteria favor experiments with poorer valid structural resolution, while RAED preserves ordering under genuine composite informativeness.The exact cyclic construction’s disagreement can grow with the number of structural families.Source: §1 · p. 3
  • A learned score-based implementation uses independent calibration for nuisance-average and tail-sensitive validity, and the benchmarks illustrate selection disagreements and tail-protection trade-offs.The studies include subsurface-flow, fluvial, and mechanistic methane-oxidation settings.Source: §1 · p. 3

2 Related Work

RAED connects experimental design to set-valued decision-making under nuisance uncertainty. Its distinguishing feature is selecting experiments by the smallest expected nonempty structural candidate set they can support under false-exclusion control.

Bayesian experimental design and model discrimination

  • Unlike information-gain and model-discrimination criteria, RAED optimizes the terminal nonempty structural candidate set while controlling false exclusion across persistent nuisance uncertainty.Related approaches include expected model elimination, which rewards experiments for expected rejection of competing models.Source: §2 · pp. 3–4

Set-valued classification and confidence sets

  • At the nuisance-average endpoint, fixed-experiment RAED reduces to classwise least-ambiguous set-valued classification when the validity and efficiency laws coincide and the same randomized decision class is used.RAED embeds that terminal decision in an outer experiment-selection problem and extends validity to persistent nuisance uncertainty.Source: §2 · pp. 3–4

Robust and ambiguity-aware design

  • RAED applies robustness to nuisance-specific false-exclusion loss and optimizes structural resolution under the resulting validity requirement, rather than robustifying an information objective.Positive δ interpolates between adverse nuisance tails and nuisance-average protection; δ = 0 is treated as pointwise validity.Source: §2 · p. 4

Comparison of statistical experiments

  • A common nuisance-independent garbling across every structural and nuisance state preserves the informativeness ordering for RAED; weaker comparisons after nuisance averaging need not.The paper also relates this benchmark to deficiency theory and connects persistent nuisance uncertainty to compound channels, confusability, and list decoding while distinguishing RAED’s one-step scientific design objective.Source: §2 · p. 4

Risk control and finite-sample calibration

  • Finite-sample calibration tools certify learned candidate-set rules against RAED’s nuisance-risk validity requirement without changing that population criterion.The scientific parameters α and δ specify the false-exclusion requirement, while ζ controls the allowed probability that certification fails.Source: §2 · p. 4

3 Resolution-Aware Experimental Design

RAED treats experiment choice as minimizing expected candidate-set size subject to false-exclusion control across persistent nuisance uncertainty. Its formulation distinguishes how nuisance states are weighted for validity from how they are weighted for resolution, and permits randomized nonempty decisions.

3.1 Structural families, nuisance states, and experiments

  • Each structural family has persistent nuisance states that govern the experiment’s observation, with separate measures for validity weighting and resolution averaging and structural weights for the resolution objective.The validity and resolution nuisance measures may coincide but need not.Source: §3.1 · p. 5

3.2 Randomized nonempty candidate-set rules

  • After observing data, RAED may randomize over nonempty structural candidate sets, with inclusion probabilities representing each family’s chance of being retained.For expected set size and familywise false-exclusion losses, these inclusion probabilities contain all information needed for optimization.Source: §3.2 · p. 5

3.3 Validity requirements

  • Validity controls false exclusion separately within each family, either at every nuisance state or through an upper-tail risk criterion under the nuisance distribution.Smaller positive δ emphasizes more adverse nuisance states, while δ = 1 recovers nuisance-average validity.Source: §3.3 · p. 6

3.4 Resolution objective and experiment choice

  • For each experiment, RAED minimizes expected candidate-set size over valid rules and compares experiments by the smallest expected structural ambiguity they can achieve under the same validity requirement.Expected candidate-set size is the primary measure; normalized resolution is used when comparisons across different numbers of families are useful.Source: §3.4 · pp. 6–7

3.5 Relationship to least-ambiguous set-valued classification

  • With nuisance-average validity and matching validity and efficiency measures, fixed-experiment RAED coincides with randomized classwise least-ambiguous set-valued classification when mandatory nonemptiness is removed.If a classical optimum is nonempty almost surely, RAED attains the same optimum value; RAED also permits randomization and requires nonempty reported sets.Source: §3.5 · pp. 7–8

3.6 Scientific validity and statistical certification

  • The false-exclusion tolerance α and nuisance-tail level δ define the scientific validity target, while ζ sets the allowed probability that finite-sample certification fails.Calibration certifies a learned rule against the validity requirement; it does not change that requirement.Source: §3.6 · p. 8

4 Information and Structural Resolution under Nuisance Uncertainty

Nuisance uncertainty can separate information about structural labels from the precision of a valid nuisance-blind structural decision. The paper exhibits this separation under cross-nuisance aliasing and shows that RAED respects a genuine composite informativeness ordering.

4.1 Cross-nuisance structural aliasing

  • Cross-nuisance structural aliasing occurs when each fixed nuisance state distinguishes structural families but changes the observation-to-label mapping, leaving one nuisance-blind rule unable to use those mappings consistently.The exact cyclic construction’s resolution gap can grow linearly with the number of structural families.Source: §4.1 · pp. 8, 11
  • The aliased experiment eI is preferred by structural and full-latent information gain and average Bayes classification, yet RAED prefers nuisance-independent eR when ε > αδ and at the pointwise endpoint.The exact Tail frontier quantifies the aliased experiment’s resolution across nuisance-tail levels.Source: §4.1 · pp. 9–10, Figure 1 · p. 11
  • Even maximin conditional mutual information prefers eI in the construction, although RAED prefers eR under the stated conditions, because nuisance-specific decoders are incompatible when the nuisance is unobserved.The paper limits this conclusion to that particular maximin conditional-MI criterion, not every robust information utility.Source: §4.1 · p. 11
  • Exact permutations are not required: near-alias regions with response laws close to a common law can also bound structural resolution, with the bound depending on nuisance-tail mass and the false-exclusion tolerance.The stated bound is nontrivial when K[1 − τ − ακδ(m)]+ > 1; in the small-mass regime m < δ, the controlling term is αδ/m.Source: §4.1 · p. 12

4.2 Composite informativeness that RAED respects

  • If one common Markov kernel simulates e2 from e1 under every composite state, every resolution level achievable with e2 is achievable with e1, so e1 cannot have a larger optimal RAED value.This is composite Blackwell monotonicity, distinguishing genuine full-state informativeness from comparisons made only after nuisance averaging.Source: §4.2 · p. 12

5 Learned RAED and Finite-Sample Validity

The section operationalizes RAED with learned score-based candidate sets and independent finite-sample calibration. It extends calibration to nuisance-tail risk and identifies the sample-size obstacle created by rare adverse nuisance states.

5 Learned RAED and Finite-Sample Validity

  • Scores are trained first, experiments are ranked on independent data, and the selected rule is calibrated on separate data for finite-sample validity.The deployed thresholds trade structural resolution against false-exclusion risk and are chosen using an independent calibration sample.Source: §5 · pp. 12–13, §5.2 · p. 13

5.1 Score-based candidate sets

  • Family-specific scores rank observations for retention, while a nonempty-set rule ensures at least one structural family is always reported.Thresholds determine which additional families are retained.Source: §5.1 · p. 13

5.2 Nested finite-sample calibration

  • Nested calibration selects the smallest candidate sets whose risk upper confidence bound is below α, without a separate multiplicity correction for each threshold.The proof relies on a single population-determined boundary point along the ordered threshold path.Source: §5.2 · p. 14

5.3 Positive-tail calibration and independent certification

  • Positive-tail calibration reduces Tail-risk certification to bounded-mean calibration and yields joint confidence across the structural families.The workflow applies for δ > 0; at δ = 0, validity is required at every admissible nuisance state and the iid calibration argument does not apply.Source: §5.3 · pp. 14–15

5.4 The statistical price of small nuisance tails

  • Rare adverse nuisance regions can remain unseen in calibration, so stringent Tail certification may require many independent nuisance states even when observed losses are small.In the construction, n independent draws miss the adverse region with probability (1 − p)^n.Source: §5.4 · pp. 15–16

6 Benchmark Construction and Experimental Protocol

The benchmarks span subsurface-flow and methane-oxidation settings, with distinct structural targets, nuisance variables, and physical experiment libraries. A common learned-RAED workflow selects experiments before independent calibration and evaluation, enabling comparisons at the level of experiment choice.

6.1 Watt: distinguishing fault architectures with active well tests

  • Watt varies the structural ontology across fault, property, and relative-permeability distinctions, then separately tests constrained sensing with shortened observation horizons and fewer remote pressure wells.The common physical ensemble includes 600 realizations and ten well tests; the constrained-sensing study fixes the three-family fault ontology.Source: §6.1 · p. 17

6.2 WCA: depositional-architecture discrimination by pressure interference

  • WCA treats the three depositional training images as structural families and their conditional facies realizations and rock properties as nuisance variation.Its ten interference tests probe different flow paths and timescales; constrained sensing varies observation duration and remote-well coverage.Source: §6.2 · p. 18

6.3 GWAE-Fluvial: one- versus two-channel discrimination

  • GWAE-Fluvial distinguishes one spanning channel from two separated channels, comparing nuisance-average RAED at δ = 1 with Tail-RAED at δ = 0.05, both at α = 0.05.The tail-versus-average comparison uses an independent ensemble of 24,000 geological realizations.Source: §6.3 · pp. 18–19

6.4 Methane oxidation: a powered certification benchmark

  • The methane benchmark distinguishes three oxidation mechanisms across 81 reactor conditions and uses 10,000 calibration nuisance states per family plus a separate 5,000-state evaluation set.At each positive tail level, calibration allocates confidence equally across mechanisms, yielding 95% joint confidence across PL, LH, and MVK.Source: §6.4 · p. 19

6.5 Common learned-RAED protocol

  • Across benchmarks, score fitting, experiment selection, calibration, and evaluation use disjoint nuisance populations, with repeated noise draws not counted as independent nuisance samples.Watt, WCA, and GWAE-Fluvial use 256-component RBF Nyström representations, while methane uses 96 components.Source: §6.5 · p. 19

6.6 Comparator criteria and downstream evaluation

  • Comparators are evaluated on the same experiment-selection population, and each selected experiment uses the same candidate-set construction, calibration, and evaluation procedure.RAED and EIG count as selecting differently only when the RAED choice falls outside the predefined Monte Carlo uncertainty set around the EIG optimum.Source: §6.6 · p. 20

7 Results

Across five benchmarks or benchmark analyses, the results show that achievable structural resolution depends on the structural target, sensing constraints, and nuisance-risk protection. Tail protection can change both the selected experiment and the ambiguity–risk balance, while methane oxidation demonstrates finite-sample certification with nontrivial resolution.

7.1 Structural granularity and achievable resolution in Watt

  • Across four Watt ontologies, RAED selects the same INJ5 pulse/falloff test, while held-out mean candidate-set size J rises from 1.421 for K = 3 to 1.599 for K = 12.The four reported J values are 1.421, 1.438, 1.525, and 1.599; these are empirical summaries, not finite-sample population certificates at the declared 5% level.Source: Figure 2 · p. 20, §7.1 · p. 21

7.2 Constrained sensing separates RAED and information gain

  • In WCA, RAED’s selections yield mean held-out J = 2.520 versus 2.658 for model-index EIG; in Watt, the means are 2.092 versus 2.111, and neither criterion dominates every comparison.WCA has seven clean disagreements among sixteen sensing combinations; in five, RAED gives smaller candidate sets, with reductions of 0.104–0.385. Watt has three clean disagreements, all favoring RAED by 0.017–0.054, but WCA also includes a clean cell favoring EIG, and some lower-J RAED choices have higher Tail risk.Source: §7.2 · pp. 21–22, Figure 3 · p. 22

7.3 Nuisance-average validity can conceal a geological hard region

  • Nuisance-average calibration in the first GWAE-Fluvial campaign met its familywise risk bounds, yet conditional false-exclusion risk was near one within the worst 1% of geological states.The evaluation familywise rates were 0.0141 for M1 and 0.0286 for M2; the 99th-percentile statewise risks were approximately 0.86 and 0.98, respectively, showing that risk can concentrate despite low population averages.Source: §7.3 · pp. 23–24, Figure 4 · p. 24

7.4 Tail protection changes the selected GWAE experiment

  • On a new GWAE-Fluvial population, nuisance-average RAED selects E01 and Tail-RAED selects E07; Tail-RAED raises J from 1.047 to 1.789 while sharply reducing upper-tail risk.The empirical upper-tail risk means fall from 0.434 to 0.014 for M1 and from 0.876 to 0.003 for M2; within the nuisance-average rule’s hard region, Tail-RAED chiefly replaces false exclusions with explicit ambiguity.Source: §7.4 · pp. 24–25, Table 3 · p. 25, Figure 5 · p. 26

7.5 Finite-sample Tail certification in methane oxidation

  • In methane oxidation, 10,000 independent nuisance states per mechanism supported 95% joint-confidence positive-tail bounds below α = 0.05, with held-out mean J = 1.750.The calibration upper bounds for PL, LH, and MVK were 0.048, 0.038, and 0.048; the empirical evaluation worst-family 5% Tail risk was 0.025. These guarantees are at fixed positive δ levels, not simultaneous across the full frontier.Source: §7.5 · pp. 26–27, Figure 6 · p. 27

8 Discussion

Across the benchmarks, experiment choice and observed resolution depend on both the sensing constraints and the declared false-exclusion validity regime. The results pair candidate-set size with exclusion behavior, and their statistical guarantees remain tied to calibration strength and the specified problem.

Empirical studies

  • Under restricted sensing, Watt and especially WCA show RAED–EIG disagreements in physical experiment choice, while GWAE tail protection changes the selected experiment and methane supports nontrivial positive-tail certification.GWAE replaces many hard-region false exclusions with explicit ambiguity, at a substantial cost in average resolution.Source: §8 · p. 27

Resolution and false exclusion

  • Across the empirical results, candidate-set size must be read alongside false-exclusion behavior: RAED can reduce ambiguity while increasing held-out Tail risk, though in other cells both improve together.Watt’s nontrivial learned rules are highly resolving, but held-out exclusion rates slightly exceed nominal levels and calibration does not support a nontrivial population certificate.Source: §8 · p. 27

Nuisance-tail protection

  • Under GWAE nuisance-average validity, most geological realizations have essentially zero conditional false-exclusion risk while a small subset is excluded almost systematically, consistent with δ = 1.With δ = 0.05, Tail-RAED selects a different experiment and reduces false exclusion in difficult states by returning ambiguous sets more often; the cost is substantially lower average resolution.Source: §8 · p. 28

Interpreting experiment selection

  • The GWAE Tail-criterion winner E07 only narrowly outranks E01, so the stronger finding is the change in calibrated resolution–risk behavior, not the selected experiment’s identity.Watt and WCA δ sweeps also show that preferred restricted-sensing designs can depend on the requested nuisance-tail protection.Source: §8 · p. 28

Statistical weight of benchmarks

  • Watt and WCA held-out evaluations are too small for positive-tail population certificates, whereas larger independent calibration populations support fixed-operating-point familywise bounds in GWAE and methane.Held-out Tail means, statewise quantiles, and hard-region analyses describe behavior in practice; formal validity statements come from calibration upper bounds.Source: §8 · p. 28

Scope

  • Validity depends on the declared nuisance sets and distributions, so GWAE conclusions apply to its constructed geological population and benchmark-specific observation models, experiment libraries, and sensing restrictions.Source: §8 · p. 28

Method and theory boundary

  • The empirical method uses a restricted learned score family, and its finite-sample population guarantees cover 0 < δ ≤ 1; the theory analyzes an ideal oracle target.The framework is one-step, with sequential designs proposed as a natural extension.Source: §8 · pp. 28–29

9 Conclusion

RAED evaluates experiments by the structural explanations they can safely eliminate under persistent nuisance uncertainty, rather than information gain alone. The empirical studies show that experiment choice and resolution–risk trade-offs depend on sensing constraints and validity protection.

Resolution-aware criterion

  • RAED values experiments by the structural explanations they can eliminate while controlling false exclusion under persistent nuisance uncertainty, and its learned formulation supports nuisance-average and positive-tail validity.Cross-nuisance aliasing can separate this objective from information gain, while common-garbling results preserve expected ordering under a genuine composite Blackwell comparison.Source: §9 · p. 29

Empirical findings

  • Under constrained sensing, Watt and especially WCA show RAED and information-based criteria selecting different physical experiments, while GWAE tail protection changes selection and replaces hard-region exclusions with ambiguity.Methane oxidation provides a complementary case where positive-tail validity is certified nontrivially with a sufficiently large nuisance-calibration population.Source: §9 · p. 29

Takeaway

  • Experiment design under partial identifiability also requires deciding which structural exclusions must remain reliable across nuisance uncertainty and how much ambiguity is acceptable at the chosen (α, δ) operating point.Source: §9 · p. 29

A Notation and Technical Preliminaries

The notation distinguishes structural families, persistent nuisance states, experiments, and the measures governing validity and resolution. The appendix also distinguishes positive-tail validity from pointwise validity.

Core objects

Measures and observation law

Rule and risk quantities

  • Γ_e(Y) is the randomized nonempty structural candidate set; g_{k,e}(y) is its family-inclusion probability, and L_{k,e}(θ) is nuisance-specific false-exclusion probability.α is the false-exclusion tolerance, δ the protected nuisance-tail mass, ζ the certificate failure probability, and J_e(g) the expected candidate-set size.Source: A Notation and Technical Preliminaries · p. 29

Tail validity

B Exact Inclusion-Marginal Representation

The appendix shows that inclusion marginals exactly represent the current RAED rule problem for its stated validity regimes. These marginals preserve expected cardinality and exclusion constraints, though they do not specify a unique distribution over returned subsets.

Measurable lifting

  • A measurable inclusion-marginal rule can be lifted to a measurable randomized whole-set rule supported on nonempty subsets.The appendix constructs the lift through a measurable selector over subset distributions matching the marginals.Source: B Exact Inclusion-Marginal… · p. 30

Finite-partition result

  • For a finite measurable observation partition, realizable marginal vectors characterize the nonempty whole-set rules; the proof represents feasible marginals as mixtures of nonempty subset incidences.Source: B Exact Inclusion-Marginal… · pp. 30–31

Exact representation

  • For pointwise, strict Tail, and nuisance-average RAED, expected cardinality and familywise exclusion constraints depend only on inclusion marginals, giving whole-subset and marginal formulations identical feasible loss fields and optimal values.Source: B Exact Inclusion-Marginal… · p. 30

Reconstruction limitation

  • Inclusion marginals fix expected cardinality and classwise retention probabilities but not coupling-sensitive summaries such as singleton frequency or the variance of |Γ|.Those summaries require an explicitly declared whole-set reconstruction; at most K + 1 nonempty subsets are needed per response-cell reconstruction.Source: B Exact Inclusion-Marginal… · p. 31

C Proofs for RAED Formulation, Aliasing, and Experiment Comparison

The appendix proves common-garbling comparisons by transporting nonempty candidate-set rules and bounding changes in validity and efficiency. It extends these comparisons to approximate garbling and one-sided deficiency.

C.1 Common and approximate garbling

  • Approximate transport increases pointwise exclusion risk by at most ε, while mixing with the full candidate set can restore the original validity level.The full-set rule is a zero-exclusion safe rule.Source: §C.1 · p. 32
  • Deficiency bounds hold above the directed deficiency, and at that endpoint when attained; exact repair also extends to a nonattained endpoint when α > 0.The corollary records these continuity conditions for the resolution–validity frontier.Source: §C.1 · p. 32
  • If both directed deficiencies are at most ε > 0, symmetric bounds hold for every δ; at ε = 0, α > 0 or attained zero deficiencies are required.At α = 0, the nonattained zero-error endpoint is not covered by continuity.Source: §C.1 · pp. 32, 34
  • Because candidate-set size lies in [1, K], the approximate-transport efficiency bound carries a factor of K − 1.The comparison concerns expectations under the transported and original experiments.Source: §C.1 · pp. 32–33
  • The risk-envelope argument transfers strict Tail validity and the nuisance-average endpoint under approximate garbling.The comparison is obtained by taking an infimizing sequence of rules.Source: §C.1 · p. 33
  • The repair mixes the transported rule with the full set to recover exact α-validity while scaling exclusion losses by 1 − λε.The result applies to pointwise, mean, and Tail risk measures.Source: §C.1 · p. 33
  • Under exact common garbling, the RAED resolution–validity frontier preserves its expected ordering.The exact claim is the ε = 0 case of the comparison.Source: §C.1 · pp. 31–33

C.2 Exact cyclic aliasing separation

This section characterizes limits on structural resolution and justifies score-based rules and calibration procedures for nuisance-average and tail risk. It also establishes validity guarantees for Monte Carlo estimates and selection among certified rules, including selected-only calibration and tail-risk claims at larger δ values.

C.2 Exact cyclic aliasing separation

  • eI can rank above eR by information, average classification, and nuisance-marginal Blackwell informativeness while its valid expected candidate-set size is K(1 −α), versus 1 for eR.Under eI, full-latent information is log K; under eR it is I_K(β) < log K, and eR's singleton rule has exclusion risk β ≤ α.Source: §C.2 · p. 35

C.3 Scaling consequence

  • For any strict Tail level, sufficiently large K admits a separation where nuisance-marginalized structural information, full-latent information, average classification, and nuisance-marginal Blackwell informativeness rank eI above eR.This is the linear-in-K scaling consequence of the exact separation.Source: §C.3 · pp. 36–37

C.4 Diffuse near-aliases

  • Under family-specific near-alias conditions, every Tail-RAED-feasible rule obeys a lower bound on expected candidate-set size, with mandatory nonemptiness supplying an independent bound of one.The theorem also gives a mixture-weighted consequence across families.Source: §C.4 · pp. 37–38

D.1 Population score geometry supporting the learned family

  • The population score geometry yields a randomized density-ratio threshold rule for classwise nuisance-average retention and represents Tail validity as robust family retention.The supporting oracle geometry requires additional compactness, domination, continuity, and strong-duality conditions for attainment.Source: §D.1 · pp. 38–39

D.2 Proof of nested nuisance-average calibration

  • Nested nuisance-average calibration reduces invalid threshold selection to failure of a pointwise upper confidence bound at a population-determined threshold, supporting simultaneous validity across claims.The proof uses a safe endpoint; it requires no union bound over thresholds along an individual nested path.Source: §D.2 · pp. 39–40

D.3 Positive-tail specialization of nested calibration

  • For positive-tail calibration, a bounded transformed loss yields a pointwise upper confidence bound on Tail risk, and nested selection validity follows under continuity of the risk path.The simultaneous statement follows from the same countable union bound.Source: §D.3 · p. 41

D.4 Nested Monte Carlo for nuisance-state loss

  • With nested Monte Carlo, Jensen's inequality makes outer nuisance-state observations conservative for Tail-risk calibration, and inner noise replicates do not increase the outer sample size.Independence is required across the outer nuisance-state observations used by the bounded-mean upper confidence bound.Source: §D.4 · pp. 41–42

D.5 Selection-aware independent certification

  • Independent final calibration gives simultaneous validity for every claim in a realized shortlist, so a deployed experiment selected from certified rules remains population-valid.The final choice may depend on calibration and ranking data.Source: §D.5 · p. 42
  • After conditioning on the independent selection stage, any later measurable choice among simultaneously population-valid rules remains valid, even if it uses calibration or ranking data.The conditional union bound establishes validity with probability at least 1 − ζ; the guarantee applies to the later selector’s available rules.Source: §D.5 · p. 42
  • Selected-only calibration certifies the deployed experiment’s still-open family and tail claims conditional on independent selection, but does not guarantee population optimality.A certified smaller-δ tail-risk statement implies claims at larger δ by monotonicity, without another union-bound term.Source: §D.5 · p. 42

D.6 Proof of the rare-tail obstruction

This section proves a rare-tail calibration obstruction, characterizes an attained least-favourable Tail-RAED score, and specifies the study’s learned-score, comparator, calibration, and benchmark procedures. It also reports held-out Watt ontology results, a GWAE-Fluvial comparison of tail-sensitive and nuisance-average RAED, and the methane benchmark’s positive-δ certification protocol.

D.6 Proof of the rare-tail obstruction

  • An all-zero sample falsely certifies the just-invalid alternative with probability at least (1 −p)^n, so requiring this probability to be at most β yields the sample-size bound and its small-αδ asymptotic.The alternative has p = δ(α + ε); the asymptotic uses −log(1 −x) ∼x as ε ↓0 in the small-αδ regime.Source: §D.6 · pp. 42–43

E.1 Attained least-favourable Tail-RAED score

  • Under dominated compact predictive families, the nonempty population Tail-RAED problem attains an optimum with zero duality gap and admits least-favourable nuisance reweightings.The characterization explains the score target: strict-tail efficiency is governed by a least-favourable reweighted predictive mixture.Source: §E.1 · pp. 43–44

F.1 Learned score systems

  • Watt and WCA use one-versus-rest logistic scores on 256-component RBF Nyström maps, while methane uses 96 features with fixed regularization and bandwidth.Cross-validation groups folds by outer nuisance state; methane uses grouped three-fold validation, and its observation-noise replicas stay within the same fold.Source: §F.1 · pp. 44–45

F.2 Comparator estimators and tie handling

  • Comparator estimates use uniform structural weighting and fixed predictive draws; methane uses 256 selection nuisance states per family and eight draws per state.Watt and WCA use 40 and 100 selection states per family, respectively; the second GWAE-Fluvial campaign compares δ = 0.05 Tail-RAED with independently ranked δ = 1 nuisance-average RAED without the external comparator suite.Source: §F.2 · pp. 45–46

F.3 Threshold calibration and Tail evaluation

  • Threshold paths are fixed from the independent selection role before calibration; positive-tail calibration uses bounded-mean Bernoulli–KL upper confidence bounds, while Watt and WCA lack small-tail population certificates.Methane uses γ = 0.05/3 and GWAE Tail calibration uses γ = 0.05/2; inner Monte Carlo calibration is conservative for the underlying nuisance-specific Tail loss.Source: §F.3 · p. 46

F.4 Selection and held-out firewall

  • Score fitting and experiment ranking precede access to final calibration data; the selected experiment's fixed score and threshold path are calibrated independently before untouched evaluation.The pools are disjoint across roles, with common random numbers confined to a role.Source: §F.4 · p. 47

G.1 Watt; G.2 WCA

  • Watt compares three fault models across four structural ontologies, while WCA treats three parent training images as structural hypotheses and conditional realizations as nuisance.The Watt ensemble reuses the same physical reservoir set across ontologies; WCA has 1,000 nuisance states per parent image.Source: §G.1 · p. 47, §G.2 · p. 49

G.3 GWAE-Fluvial

  • GWAE-Fluvial defines a two-family target distinguishing one connected spanning channel from two separated spanning channels.The prospective ensemble uses interpretable channel-geometry and petrophysical parameters.Source: §G.3 · p. 50
  • The GWAE-Fluvial comparison ranks Tail-RAED at δ = 0.05 against nuisance-average RAED at δ = 1 using selected experiments and a common untouched evaluation population.Each structural family contributes 12,000 nuisance states simulated under all nine tests; ranking, calibration, and evaluation use separate samples.Source: §G.3 · pp. 51–52

H Methane-Oxidation Benchmark Construction and Certification Protocol

  • 95% joint confidence is provided across PL, LH, and MVK at each fixed positive δ, while δ = 0 results remain empirical.Positive-δ guarantees apply separately at each fixed operating point, not simultaneously across the δ grid.Source: H Methane-Oxidation Benchmark… · p. 53

I.1 Watt ontology summary

  • INJ5 pulse/falloff is selected across all four Watt ontologies at every δ, while finer ontologies retain higher normalized resolution across the empirical frontier.Positive-δ calibration could not certify a nontrivial population-valid rule, so these results are an empirical frontier rather than safe-endpoint deployment.Source: §I.1 · pp. 53–54

I.2 Complete restricted-observation results at δ = 0.05

Under restricted sensing, RAED and EIG select different experiments, with resolution–risk tradeoffs that vary across benchmarks. Tail protection can also change selected designs and outcomes in the fluvial and methane studies.

I.2 Complete restricted-observation results at δ = 0.05

  • In Watt and WCA, RAED-selected experiments sometimes yield smaller held-out candidate sets but higher worst-family Tail exclusion; none of these restricted-sensing rules has a population-validity certificate.The comparisons therefore report candidate-set size alongside held-out Tail risk; Watt also has disagreements with lower Tail risk at two sensing levels and higher risk at another.Source: Table 14 · p. 55, §I.2 · p. 56, Table 15 · p. 56

I.3 Dependence of the selected restricted design on δ

  • Across seven δ operating points, the selected design changes at 3 of 16 Watt sensing levels and 10 of 16 WCA levels.The stronger dependence in WCA indicates greater sensitivity of the preferred sensing strategy to nuisance-tail protection.Source: §I.3 · p. 57

I.4 GWAE-Fluvial Tail-RAED comparison

  • Tail-RAED selects the E07 staged-rate pair, while nuisance-average RAED selects E01, the symmetric-bank test.The ranking uses an independent population before final selected-only calibration.Source: §I.4 · p. 57, Table 17 · p. 57
  • Calibration uses 5,000 independent nuisance states per family and yields 95% joint confidence across M1 and M2 for the fixed operating point.The reported upper bounds are 0.048330 for M1 and 0.049103 for M2 under Tail-RAED, and 0.025999 and 0.049341 under nuisance-average RAED.Source: §I.4 · p. 57
  • Tail protection raises mean candidate-set size by 0.741021 (1.788292 vs. 1.047271) while lowering pooled empirical false-exclusion risk (0.000419 vs. 0.034099).With K = 2, ambiguity frequencies are 0.788292 for Tail-RAED and 0.047271 for nuisance-average RAED.Source: §I.4 · p. 58, Table 18 · p. 58
  • In the nuisance-average rule’s worst 5% of geological states, Tail-RAED false-excludes M1 on 1.38% of draws and M2 on 0.29%, versus 43.38% and 87.58%.Tail-RAED instead returns ambiguity on 98.62% of M1 draws and 99.71% of M2 draws in those states.Source: §I.4 · p. 58

I.5 Methane frontier

  • For methane oxidation, MOX048 is selected at every declared δ; each positive-δ operating point has a 95% joint-confidence familywise positive-tail population guarantee across PL, LH, and MVK.The guarantees are separate by operating point, not simultaneous across the δ grid; δ = 0 is only an empirical diagnostic.Source: §I.5 · p. 58
  • At δ = 0.05, methane calibration upper bounds are 0.047649 for PL, 0.037982 for LH, and 0.047738 for MVK; certification increases held-out mean candidate-set size by 0.061716.The normalized resolution is R = (K − J)/(K − 1).Source: §I.5 · p. 59
Loading 2609.03686v1…