PaperlayerView exact revision on arXiv

Source-linked AI summary

The Cost of a Physics Prior Is Bounded by the Ablation Gap

Boris Kriuk

arXiv:2608.21059v1cs.AI

TL;DR

Constraint-cost reports lack a stable interpretation when they ignore the free features and validation split. The paper proves an ablation-gap bound, tests it on wildfire severity, and finds that shielding and protocol choice substantially change reported prices while a calibrated floor filters uninterpretable results.

  • Problem

    Shape-constrained and physics-informed learning often reports the accuracy cost of enforcing a prior as if it were a property of the prior itself.

  • Method

    The paper nests the ablated hypothesis class inside the constrained class, tests the resulting ablation-dominance sign check across validation protocols, and calibrates a resolution floor from certified nesting inversions.

  • Results

    The same monotone prior costs 0.0473 macro-F1 with coordinates left free and 0.3470 without them, while four headline price cells fall below the 0.0220 macro-F1 resolution floor.

  • Takeaways & Limitations

    Constraint-cost claims should report the price, ablation gap, resolution floor, free feature set, and validation protocol together.

Abstract

from arXiv · show

Shape-constrained and physics-informed learning reports an accuracy cost of enforcing a prior and treats it as a property of the prior. We show it is mostly a property of the free features and the validation split. Let P be the excess risk of restricting a hypothesis class to functions with a shape constraint on features S, and D the excess risk of the ablated model that ignores S. Because a function constant in x_j is both non-decreasing and non-increasing in x_j, the ablated class is contained in the constrained class, so 0 <= P <= D for every risk functional, with no convexity, smoothness, or realizability assumption. Empirically the bound is a sign test: a constrained model must never be beaten by its own ablation. We instantiate it on an ordinal wildfire-severity task (N = 26,681, K = 3) with hard monotone constraints on four meteorological drivers, coordinates left free, and a validation ladder from i.i.d. resampling to 2-degree spatial blocking. Coordinates act as a shield: alone they recover 92.9% of the full model's macro-F1 under spatial blocking, collapsing D from 0.1288 to 0.0427; the same prior costs 0.0473 shielded and 0.3470 unshielded, a ratio of 7.3 with identical physics. Because D is protocol-dependent it does not transfer: coarsening blocks from 1 to 10 degrees drives D from 0.0942 to 0.0050, leaving two configurations unidentifiable a priori. Inversions of the certified nesting bound the pipeline's additive resolution: over 318 comparisons they give a self-calibrating floor of 0.0220 macro-F1, below which no reported price is interpretable, including four cells in our own headline grid. Cost and compliance are independent: the unconstrained model violates the prior at rate 0.48-0.49 while enforcing it costs 0.0473. We give a two-fit screen that rejects unidentifiable experiments before a constrained model is trained.

I. INTRODUCTION

The paper reframes reported constraint costs as quantities shaped by free features and validation protocols, using ablation to expose a structural bound. It extends this observation into practical checks for reporting, calibration, and experiment design.

  • Motivation and structural observation: The paper argues that reported prior costs should be read alongside ablation gaps rather than treated as intrinsic properties of the physics.The same comparison is already available in experiments that report ablation or feature-importance baselines.
  • Motivation and structural observation: Ablated hypotheses that ignore constrained features remain admissible under shape constraints, so the constrained optimum cannot be worse than the ablated optimum.This nesting yields a ceiling on the cost of imposing the constraint and requires no special assumptions about the risk functional.
  • Scope of the result: The framework covers hard and penalized constraints, including monotonicity, convexity, Lipschitz, and unimodality restrictions.Its generalization rests on the fact that functions constant in constrained coordinates satisfy these properties.
  • Empirical implications: Coordinates shield the headline wildfire experiment: latitude and longitude recover 92.9% of full-model macro-F1 and make the identical monotone prior appear 7.3× cheaper.The shielded and unshielded prices are 0.0473 and 0.3470, respectively, under spatial blocking.
  • Empirical implications: The study recommends reporting the ablation gap and a resolution floor, because protocol-dependent ceilings can make small reported prices uninterpretable.The paper also presents a pre-training screen for experiments whose ablation gaps fall below the calibrated resolution.
  • Empirical implications: Accuracy cost and prior compliance are independent: the unconstrained model violates the prior at rate 0.4795–0.4825 while enforcement costs 0.0143–0.0549.The authors caution that shape constraints may still provide extrapolation, auditability, and regulatory defensibility beyond in-distribution accuracy.

A. Monotone and shape-constrained learning

The paper situates monotone and shape-constrained learning within physics-informed and structured prediction work, then formalizes risks, ordinal scores, feature constraints, and ablation closure for its analysis.

  • Prior work and formal setup: Monotonicity has developed from early inductive-bias work into monotonic networks, partially monotone models, lookup tables, deep lattice networks, and certified architectures.The paper also notes that monotone splits are standard in production gradient boosting.
  • Prior work and formal setup: Physics-informed learning embeds conservation laws, scaling relations, or prescribed sensitivities into predictive models, often reporting scientifically plausible parameters or preserved extrapolative behavior.The paper references permafrost infrastructure modeling and POSEIDON as examples.
  • Prior work and formal setup: Spatially autocorrelated data motivate validation beyond random k-fold, because random resampling can inflate apparent skill and alter the interpretation of constraint costs.The paper reproduces this concern in its validation comparisons.
  • Prior work and formal setup: The formal risk functional permits expected losses, negated macro-F1, AUC deficits, quantiles, and cross-validation estimates, with lower values preferred.This lets the theorem apply to metrics that are not decomposable examplewise, including macro-F1.
  • Prior work and formal setup: The ordinal model uses cumulative scores whose class ordering is enforced by isotonization, while shape constraints are imposed on each cumulative score and preserved in the feature coordinates.The label is decoded from the induced class probabilities.
  • Prior work and formal setup: The constrained feature set is specified by a sign pattern, while the S-blind class contains hypotheses invariant to all coordinates in S.Ablation closure requires this class to be nonempty.
  • Prior work and formal setup: For the histogram gradient-boosted ensembles used here, avoiding splits on constrained coordinates constructs an S-blind model satisfying any per-feature monotonicity specification.The paper states that this includes contradictory specifications.
  • Prior work and formal setup: The ablation gap is the quantity applied papers report when describing the performance change from removing a constrained feature, and it is non-negative by nesting.The paper separately defines violation rates for assessing shape compliance through internal scores and deployed preprocessing.

IV. MAIN RESULT

The main result follows from hypothesis-class nesting: functions that ignore constrained features satisfy the corresponding shape constraints, bounding the constrained risk by the ablated risk for every risk functional.

  • IV. MAIN RESULT: Every function constant in the constrained coordinates is admissible under any sign pattern because its dependence on each constrained coordinate is weakly constant.This is the degenerate-admissibility lemma.
  • IV. MAIN RESULT: Theorem 9 states that, under ablation closure, the excess risk of a shape-constrained class is at most the ablation gap for every risk functional.The proof uses only the ordering of infima over nested hypothesis classes.
  • IV. MAIN RESULT: The theorem is intentionally elementary; its practical contribution is connecting a routinely computed ablation quantity to the cost of a constraint.The paper develops this connection through later corollaries and empirical calibration.

A. The operational form: ablation dominance

Ablation dominance turns the theoretical ceiling into a direct sign check, while the paper extends the bound across generalized constraints, soft penalties, sharpness regimes, and feature- and protocol-dependent empirical cases.

  • A. The operational form: ablation dominance: The operational form requires that the constrained model not be beaten by the model discarding the constrained features.It needs neither the unconstrained F1 nor subtraction of two estimates against a common reference.
  • A. The operational form: ablation dominance: The sign check compares two directly tabulated quantities and is numerically better behaved than differencing both against a common reference.This makes it computable from an existing experiment harness without additional training.
  • B. How far the bound generalizes: The bound applies to degenerate-admissible families including arbitrary partial-order monotonicity, convexity, concavity, Lipschitz, unimodality, bounded variation, sensitivity, and dominance constraints.Functions constant in S satisfy each listed property.
  • B. How far the bound generalizes: Soft penalties with zero penalty on S-blind functions obey the same ceiling independently of the penalty weight, so tuning λ cannot exceed the constrained features’ ablation value.This covers penalized objectives commonly used in physics-informed pipelines.
  • B. How far the bound generalizes: The bound is sharp: a binary construction attains P(σ) = ∆({1}) = 1 under a monotonicity constraint opposing the Bayes labels.Thus the ceiling cannot be improved in general.
  • B. How far the bound generalizes: A measured price near the ablation gap indicates anti-alignment with the mechanism, whereas a much smaller price is only weak evidence for the prior.An ablation gap near zero instead informs the feature set rather than the physics.
  • B. How far the bound generalizes: In the wildfire corpus, P/∆ is 0.82 unshielded and 0.37 shielded, while coarse blocking yields ∆ = 0.0050.These regimes show that the same prior’s apparent cost depends on shielding and validation resolution.

V. CONSEQUENCES

Protocol choice and free features determine the observable cost of a constraint. A free-feature shield can make the prior cheap regardless of whether it is correct, while changing the validation protocol can reverse measured costs.

  • Protocol dependence: PΠ(σ) ≤ ∆Π(S) for every protocol, so a sufficiently small ablation gap bounds the cost of every prior supported on S.The bound does not transfer across protocols: PΠ provides no information about PΠ′ beyond their ablation gaps and joint behavior.
  • Protocol dependence: 0.0456 at 1° blocks became −0.0139 at 10° blocks for the same prior and corpus, showing a split-driven sign reversal.Thus costs reported under random and blocked splits need not measure the same quantity or indicate disagreement.
  • Shielding: ∆(S) ≤ ε whenever an S-blind predictor is within ε of the unrestricted optimum, and enlarging free features cannot increase ∆(S).This is the shielding mechanism: proxy features can reconstruct the signal carried by constrained features.
  • Shielding: A small measured price with a free proxy set is evidence that constrained features were unnecessary, not evidence that the model respects the physics.Here the shield is {lat, lon}; adding insolation as a fifth constrained driver increased the estimated ablation gap from 0.1288 to 0.1399 under i.i.d. splitting.
  • Sign patterns: If the ablation gap is below the noise floor, accuracy cannot adjudicate competing prior signs; extrapolation behavior or external theory is required.The paper reports the empirical spread over sign patterns and decomposes bound violations into elementary inversions.

D. Slack as an instrument

Certified class nesting turns empirical order violations into a calibration instrument. The resulting slack estimates pipeline error and supports a pre-training screen for experiments whose constraint price is unidentifiable.

  • Certified comparisons: A certified comparison pairs nested hypothesis classes whose population metric ordering must satisfy F1(B) ≥ F1(A).Its inversion measures departure from the ordering that the experiment is obliged to satisfy.
  • Slack as calibration: ˆs lower-bounds the additive pipeline error in the reported metric across a grid of certified comparisons.The bound reflects finite-sample estimation and constrained-solver suboptimality rather than only resampling uncertainty.
  • Slack as calibration: Any |ˆPc| < ˆs is unresolvable, and ˆ∆c < ˆs makes the price of any prior on S unidentifiable before constrained training.The admissible interval [0, ˆ∆c] then lies inside the estimated pipeline noise.
  • Pre-training screen: Algorithm 1 fits the unconstrained model and its S-ablation first, stopping when ˆ∆ < ˆs; otherwise it proceeds to constrained training.The screen therefore requires no constrained solver for experiments it rejects.
  • Caveat: ˆs = 0.0220 is treated as a lower bound on pipeline resolution because the reported calibration omits seed variance and multiplicity correction.A seed sweep could increase the slack and disqualify additional cells.

E. Compliance is not price

Constraint cost and physical compliance are distinct quantities. The model zoo maps certified nested classes to concrete feature and sign configurations, while unmatched feature additions cannot be interpreted as constraint prices.

  • Independence: P(σ) and violation rate can vary independently: zero constraint cost can coexist with violation approaching one, and zero violation can coexist with positive cost.The construction uses redundant or irrelevant constrained features to separate predictive risk from pointwise compliance.
  • Independence: A high violation rate with a small price is generic when constrained features are partially redundant, and the paper measures this dissociation.The surrounding model correspondence uses one shared ordinal ensemble architecture while varying feature sets and monotonic specifications.
  • Certified model zoo: Four families of certified ceilings provide the falsifiable empirical tests, including ceilings for feature ablations, sign variants, and nested constraint classes.These ceilings follow by applying the general ablation bound to the corresponding fitted-model classes.
  • Certified model zoo: The sign-spread ceiling is maxctx sign F1 − minctx sign F1 ≤ F1(MIB) − F1(MONO sc).The comparison bounds accuracy differences among sign choices by the gap between the auxiliary-channel model and its channel-blind counterpart.
  • Model-zoo caveats: MIB versus MC is not a certified constraint cost because the classes differ in features and no feature-matched unconstrained reference was fitted.Their differences combine the D-prior with the added channel, so MIB exceeding MC is not an inversion.
  • Model-zoo caveats: When the auxiliary channel is absent, MIB, MIB fp, and MIB fm must equal MONO sc bytewise because constraining a nonexistent feature is vacuous.The implementation asserts this degeneracy and reports that the assertion passes in the tested protocols.

VII. EXPERIMENTAL DESIGN

The experiments calibrate the theorem on a georeferenced ordinal wildfire-severity corpus using hard monotone constraints, geographic shielding, and fold-wise sign and auxiliary-channel screening. Corpus structure limits the validation ladder mainly to unblocked and spatially blocked comparisons.

  • Study scope: The study calibrates Theorem 9 rather than introducing a new predictive method, using an environmental task with uncontroversial monotone physical priors.The fitted task is designed to measure ablation gaps, constraint prices, and empirical slack.
  • Corpus and features: N = 26 681 records form a K = 3 ordered wildfire-severity target, with drivers D = {temp, rh, wind, precip} and geographic shield G = {lat, lon}.The majority-class macro-F1 baseline is 0.3132.
  • Validation design: The absence of a parsable date and near-surjective coordinates leave two informative validation rungs: unblocked and spatially blocked.Exact-coordinate grouping removes little information relative to i.i.d. resampling, while temporal validation and calendar harmonics are unavailable.
  • Constraint design: The textbook monotonic pattern is σnv = (+1, −1, +1, −1), while σsc uses fold-wise mean finite-difference effects from the unconstrained MC model.Training-fold-only screening avoids leakage and exposes non-identified signs through clustered bootstrap intervals.
  • Constraint design: σsc matched σnv bytewise in 9 of 15 headline-configuration training folds, so MONO sc was fitted as MONO naive in those folds.Signature deduplication detected the duplicated fits.
  • Auxiliary channels: Only insolation survived auxiliary-channel screening, which admits candidates only when they are not reconstructible from the four meteorological drivers.Candidates are tested with a quadratic regression in D using R2 ≤ 0.90 and residual relative standard deviation ≥ 0.10.

D. Validation ladder

The validation ladder moves from i.i.d. resampling to coordinate and spatial blocking, revealing protocol-dependent performance, ablation ceilings, and a certified dominance check for constrained models.

  • Validation protocols: Three protocols comprise the ladder: i.i.d. stratified 5-fold, exact-coordinate grouping, and spatial blocking with chosen angular block sizes.Grouped folds use stratified group k-fold; the headline grid uses 429 blocks at 2°.
  • Validation protocols: 0.8316 to 0.6029 macro-F1: the unconstrained model drops across i.i.d. to 2° spatial splits, a split-leakage term of +0.2287.Under 10° blocks, macro-F1 reaches 0.5218, with a term of +0.3098.
  • Certified headroom: 0.1288 to 0.0427: the certified ceiling ˆ∆(D) shrinks from i.i.d. to 2° spatial blocking, forcing every prior’s admissible cost range to shrink.The intermediate point-grouped value is 0.1165, making the overall change a factor of 3.0.
  • Certified headroom: All fifteen constrained-versus-GEO ONLY comparisons lie to the right of zero, with margins from +0.0029 to +0.0941.This is the empirical ablation-dominance test required by Corollary 11.
  • Cost accounting: 0.367, 0.476, and 0.419: the normalized MONO sc price is the fraction of driver value destroyed across i.i.d., point, and spatial protocols.The corresponding unshielded ratios are 0.821, 0.822, and 0.694, showing that the feature set changes the interpretation of cost.

C. Shielding, measured

Coordinates shield the monotone drivers from much of the measured cost, but both shielding and validation protocol alter the available headroom and can make prices unidentifiable.

  • Shielding: 92.9%: coordinates alone recover this fraction of the full model’s macro-F1 under spatial blocking and beat the four drivers together by +0.0572.Under i.i.d. splitting, coordinates recover 84.5% of the full model.
  • Shielding: 0.0473 versus 0.3470: the same screened monotone prior costs far less with coordinates available than with the unshielded driver-only feature set.The ratio is 7.34 under i.i.d.; the physics is unchanged and only the free feature set differs.
  • Shielding: 0.367 to 0.821: the normalized price more than doubles from shielded to unshielded features, so reporting the free feature set is necessary for interpreting constraint cost.The unshielded value is close to the anti-alignment regime described by Proposition 15.
  • Protocol dependence: 0.0050 at 10° blocks: the ceiling becomes a fifth of ˆs, making the last two measured-price entries unresolvable.Across the block-size sweep, the MONO sc prices are 0.0456, 0.0179, 0.0229, and −0.0139.
  • Configuration sensitivity: 0.1288 to 0.1399: promoting insolation from auxiliary channel to fifth driver raises the ceiling under i.i.d. splitting, while the shielded normalized price falls to 0.308.The unshielded normalized price falls to 0.765 in the same configuration.
  • Configuration sensitivity: 0.451, 0.537, and 0.534: refining the label set to K = 4 raises MONO sc’s normalized price across the validation ladder relative to K = 3.The ceiling remains essentially unchanged, while driver signs become identifiable in this configuration.

E. Sign identification, and why its stakes are bounded

Sign identification depends partly on the blocking protocol and label resolution, while the accuracy consequence of choosing a sign is bounded by the ablation gap. In the auxiliary-channel analysis, certified nesting provides a benchmark for interpreting sign sensitivity and exposes pipeline-resolution limits.

  • Sign identification: 88.3% of driver-folds match the textbook sign, but only 21.7% have intervals excluding zero.Temperature and relative humidity agree in 15/15 folds, wind in 14/15, while precipitation agrees in 9/15 and reverses sign in 6/15 folds.
  • Bounded stakes: 0.0427 under spatial blocking and 0.1288 under iid bound the accuracy at stake in resolving the sign dispute.At 10° blocks the bound falls to 0.0050, making the dispute unresolvable on accuracy grounds.
  • Protocol dependence: Sign identifiability changes with blocking: temperature is identified in 9/15 folds at 1° blocks, 5/15 at 2°, 4/15 at 5°, and 6/15 at 10°.This makes identification partly a property of the analyst’s blocking choice rather than solely of the underlying physics.
  • Compliance versus price: 0.4795 under iid and 0.4825 under spatial blocking are the unconstrained model’s textbook-prior violation rates, near the 0.5 coin-flip level.The measured price of enforcing the same prior is 0.0143–0.0555, so violation and price can be high and low simultaneously.
  • Pipeline compliance: 0.0000 end-to-end own-declaration violations hold for all three MIB arms across protocols and gate-open configurations.The residualizer therefore does not reintroduce sign violations after preprocessing.
  • Auxiliary-channel sensitivity: 0.0295 is the largest reported bound violation, decomposing into certified nesting inversions rather than an additional resolution term.The paper reports 0.0295 = 0.0199 + 0.0096 for b1/spatial and 0.0254 = 0.0034 + 0.0220 for ctx/spatial.

H. Calibrating the slack

Across 318 certified comparisons, empirical inversions calibrate a 0.0220 macro-F1 resolution floor, rendering smaller reported prices uninterpretable and enabling pre-training futility screening.

  • H. Calibrating the slack: 0.0220 exceeds the narrower ablation-dominance floor of 0.0154, showing that the full certified ordering captures additional empirical gaps.The narrower floor is restricted to ablation-dominance pairs.
  • H. Calibrating the slack: At 10-degree blocks, constrained MONO naive beats its unconstrained superclass MC by 0.0202 despite an interval of [−0.0128, +0.0529].The same cell has ˆ∆ = 0.0050, so the comparison is below the measurable ablation gap.
  • H. Calibrating the slack: Four spatial headline price cells below 0.0220—0.0143, 0.0179, −0.0096, and −0.0130—are reported as uninterpretable.The affected cells are MONO naive, MONO sc, MIB, and MIB fp.
  • H. Calibrating the slack: Two configuration-protocol cells, b5/spatial with ˆ∆ = 0.0184 and b10/spatial with ˆ∆ = 0.0050, are unidentifiable before constrained training.Algorithm 1 would terminate these experiments after two unconstrained fits.

I. The screening-leak probe returns a null

The screening-leak probe reproduces the reference configuration bytewise, while the paper recommends reporting ablation gap, price, slack, free features, validation granularity, and label resolution.

  • I. The screening-leak probe returns a null: The glob screening variant reproduces ctx bytewise across all three protocols because insolation was admitted in all 15 training folds.The probe tests leakage in channel admission only; sign-pattern screening remains fold-specific.
  • I. The screening-leak probe returns a null: The b2 configuration also reproduces ctx bytewise on all 33 model-protocol cells under seed 0.This confirms harness determinism at that seed, not seed-to-seed variability.
  • A. What should be reported: Every accuracy-cost claim should report the triple (ˆP, ˆ∆, ˆs) together with the free feature set.The paper identifies the ablation as already implemented, with the missing step being the comparison.
  • A. What should be reported: Coarsening spatial blocks from 1 to 10 degrees changes temperature identification from 6/15 to 9/15 folds without changing the data.Bootstrap clustering granularity therefore belongs alongside claims that a driver effect sign is identified.
  • C. Why constraints can still be worth imposing: The paper distinguishes interpolation cost from extrapolation and auditability: coordinates recover 92.9% of full-model accuracy but cannot transfer outside the sampled domain.The constrained arms also retain an end-to-end violation certificate, VRown = 0.
Loading 2608.21059v1…