PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does Perturbation Consistency Add Information Beyond Softmax Confidence? Theory and Evidence for the Decision Consistency Framework

Amit Kalitaa, Moirangthem Tiken Singh

arXiv:2610.36640v1cs.LG

TL;DR

The paper asks whether perturbation consistency reveals more about prediction correctness than clean softmax confidence. It analyzes DCF through a margin-displacement inequality and finds consistency weaker than MSP on clean data but informative under noise and adversarial perturbations.

  • Problem

    The paper asks when perturbation consistency adds information about prediction correctness beyond clean softmax confidence.

    Source: §1 · p. 2

  • Method

    DCF measures label stability and confidence deviation under input operators, while a margin-displacement inequality characterizes when consistency can add information and yields a no-flip certificate.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across 201 evaluations, CS underperformed MSP for clean-data misclassification detection in every model family, while adding information under noise corruptions and adversarial perturbations.

    Source: §7 · p. 19

  • Takeaways & Limitations

    On OCTMNIST, prior correction recovers about 40% of the large negative consistency-accuracy gap, indicating that class prior partly accounts for it.

    Source: §7 · p. 19

    Most experiments use image-scale data and mild operators; label preservation was screened automatically on three datasets without human validation.

    Source: §6.3 · p. 18

Abstract

from arXiv · show

Perturbation consistency scores judge whether a prediction survives small input changes. The Decision Consistency Framework (DCF) scores each image by label stability (LS) and confidence deviation (CD) under six mild operators, combines them into a consistency score (CS), and summarises a model by the gap between accuracy and mean consistency (CAG). We ask when these scores carry information that the clean softmax output lacks. Across 201 evaluations with three seeds per configuration, CS detected misclassifications on clean data less well than the maximum softmax probability (MSP) in every model family. For frozen CNNs, the deficit was 0.057 in area under the ROC curve (AUROC; 95% CI 0.043 to 0.073), at 8 to 15 times the compute. A label flip requires the logit margin to be at most the square root of two times the logit displacement, and at a fixed margin, the displacement predicted errors at AUROC 0.515. The same inequality gives a label-free split-conformal no-flip certificate, never violated in 84 runs. Consistency did add information where confidence is misled: under noise corruptions of CIFAR-10-C, it beat the margin, and under projected gradient attacks, it separated clean from attacked inputs with AUROC 0.83 against 0.37 for MSP at a budget of 2/255, as pre-specified. The gain under noise did not transfer to unseen shift families. The large negative CAG on OCTMNIST persists under full fine-tuning and in-domain training and is partly a class-prior effect: correcting the prior recovers about 40% of the gap.

1. Introduction

This paper asks when perturbation consistency adds information beyond clean confidence, and tests that question with theory and pre-specified experiments. Consistency is mostly redundant on clean data, but can help when confidence is misled.

1. Introduction

  • DCF applies six mild operators per image, records label stability and confidence deviation, combines them into a consistency score, and summarizes models with CAG.The consistency-accuracy gap is the difference between clean accuracy and mean consistency.Source: §1 · p. 1
  • Consistency checks require extra forward passes, so their cost is justified only if they add information absent from the clean output.The paper compares them directly against confidence scores computed from one forward pass.Source: §1 · p. 2
  • A label flip requires the clean logit margin to be at most √2 times the logit displacement, explaining why consistency can be redundant beyond clean confidence.Consistency can add information only if displacement relates to correctness at a fixed clean output; that dependence was negligible on clean data.Source: §1 · p. 2
  • The same inequality yields a label-free split-conformal no-flip certificate whose bound held at every tested level in every run.The rule requires neither a Lipschitz constant nor a shared perturbation budget.Source: §1 · p. 2
  • Across model families, consistency scores were below MSP for clean error detection, while offering gains under noise corruptions and adversarial inputs where confidence is misled.The gains did not transfer to unseen shift families, and an adaptive attacker could evade the detector at the cost of lower attack success.Source: §1 · p. 2
  • The large negative CAG on retinal imaging persisted after full fine-tuning and in-domain training, and prior correction recovered part of the gap.The paper attributes part of the pattern to a class-prior shift between the official splits.Source: §1 · p. 2

2. Related Work

The related work positions single-pass confidence scores as the baseline for perturbation-based reliability methods, while identifying distribution shift, certification, adversarial detection, and label shift as distinct settings. These literatures motivate comparisons rather than establishing that any one score dominates across settings.

2.1. Misclassification detection from a single forward pass

  • Single-pass scores such as MSP, margin, entropy, and energy are strong misclassification-detection baselines, and elaborate alternatives rarely outperform them.Post-hoc methods can often address remaining MSP weaknesses.Source: §2.1 · p. 3

2.2. Perturbation consistency and test-time augmentation

  • Perturbation-based methods use prediction changes for robustness, training, adaptation, or detection; DCF instead uses agreement and confidence change as a reliability score.Unlike test-time augmentation, DCF does not average predictions.Source: §2.2 · p. 3

2.3. Uncertainty under distribution shift

  • Under distribution shift, uncertainty estimates degrade, and a clean-data score must outperform clean accuracy itself to predict shifted behavior.A correlation with robustness alone is insufficient.Source: §2.3 · p. 3

2.4. Margins, Lipschitz bounds and conformal guarantees

  • Margin-based certificates use Lipschitz bounds, whereas the proposed conformal approach calibrates a quantile of measured displacements without a Lipschitz constant.Conformal guarantees are finite-sample and distribution-free under exchangeability.Source: §2.4 · p. 3

2.5. Adversarial detection and adaptive attacks

  • Adversarial detection needs evaluation against adaptive attacks because detectors that succeed against fixed attacks can fail when the attacker knows the detector.Non-differentiable components such as JPEG may require BPDA approximations during attack.Source: §2.5 · p. 3

2.6. Label shift

  • When training and test class priors differ but class-conditionals do not, posteriors are biased toward the training prior, making label shift a candidate explanation for confident errors.Prior re-estimation can be performed without labels.Source: §2.6 · p. 4

3. Decision consistency: definitions and theory

The framework represents perturbation behavior through label stability, confidence deviation, and a combined score, then relates flips and added detection information to margins and logit displacements. It also derives a split-conformal no-flip certificate and specifies the evaluation costs and conditions.

3.1. Notation

  • The notation defines clean logits, probabilities, prediction, MSP, margin, perturbation operators, displacement, and correctness for the image-level quantities.The operators may be random, with randomness drawn independently for each image.Source: §3.1 · p. 4

3.2. The DCF quantities

  • Label stability is the fraction of operators that preserve the clean prediction, while confidence deviation measures mean absolute change in top-class confidence.The consistency score combines these quantities, but a single score does not identify their individual values, so both are reported alongside it.Source: §3.2 · p. 4

3.3. What a flip implies about the margin

  • For every image and operator, a label flip implies the clean margin is at most √2 times the logit-displacement norm.The implication requires no assumption about the classifier; the probabilistic Lipschitz bound discussed here is vacuous and requires a shared budget across operators.Source: §3.3 · p. 5

3.4. A split-conformal no-flip certificate

  • The split-conformal rule certifies an image when its clean margin exceeds a threshold calibrated from worst-case scaled displacements, bounding the probability of a certified image flipping by β.It uses no labels and only the clean margin at test time, but its marginal guarantee depends on exchangeability and may fail under distribution shift.Source: §3.4 · pp. 5–6

3.5. When can consistency add information?

  • Consistency adds no error-detection information when displacements are conditionally independent of correctness given clean logits; then the posterior using logits and consistency equals the logits-only posterior.At fixed clean output, added information therefore requires displacement to differ with correctness.Source: §3.5 · p. 6

3.6. Evaluation pipeline

  • Computing LS, CD, and CS for N images costs N(K + 1) forward passes, while the certificate needs one forward pass per test image after calibration.The certificate's calibration threshold is computed separately.Source: §3.6 · p. 6

4. Experimental protocol

The protocol evaluates DCF across four image datasets and five model groups, using fixed perturbations and comparisons with confidence and perturbation-based baselines. It also tests shifted, adversarial, and out-of-distribution inputs with pre-specified analyses and configuration-level statistics.

4.1. Datasets

  • The evaluation covers three natural-image datasets and OCTMNIST, with disjoint evaluation and calibration sets; OCTMNIST uses its validation split for calibration.Accuracy and ECE are computed on the full test split, while DCF quantities are computed on the evaluation set.Source: §4.1 · p. 6

4.2. Models and adaptation

  • Five model groups span frozen and fully trained CNNs, vision transformers, a model zoo, and adaptation controls on OCTMNIST and Intel-Scene.The adaptation controls include longer head training, full fine-tuning, and training ResNet-18 from random initialization; OCTMNIST fine-tuning used a stratified 20,000-image subset.Source: §4.2 · pp. 7–8

4.3. Perturbation operators and audit

  • Six mild operators are audited for input distance and prediction flips; rotation is a null operator on OCTMNIST, and a vision-language screening check preserved the original class on all 420 images.The audit found that input-distance spread across images was small relative to differences between operators.Source: §4.3 · p. 8

4.4. Baselines and evaluation measures

  • Misclassification detection compares DCF scores with single-pass confidence scores and other perturbation-based scores, using AUROC and AURC as primary measures.Under shift, the evaluation also measures shifted-data AUROC, errors of label-free accuracy estimators, and model-level rank correlations with shifted accuracy; cost is measured in forward passes and wall-clock time.Source: §4.4 · pp. 8–9

4.5. Shift, adversarial and out-of-distribution tests

  • Shift tests include generated and official CIFAR-10-C corruptions and a calibration-selected operator-strength sweep; adversarial tests use PGD, while OOD tests use SVHN and CIFAR-100.The adaptive attack adds a stability term to its objective, and the attacker uses a noise draw not shared with the defender.Source: §4.5 · p. 9

4.6. Statistics, pre-specification and reproducibility

  • Comparisons use a hierarchical bootstrap over configurations, seeds, and images, with the configuration as the unit of analysis; five predictions and parameter grids were specified before adversarial and OOD results were examined.The comparisons report paired differences with 95% intervals, Cohen’s dz, and Holm-corrected Wilcoxon signed-rank tests.Source: §4.6 · p. 9

5. Results

Across clean-data evaluations, consistency scores generally underperform single-pass confidence baselines, despite variation in consistency measurements across models and operators. The comparisons also show that consistency can largely re-express confidence while requiring additional computation.

5.1. Reference measurements

  • On CIFAR-10, transformers had CS 0.945 to 0.961 versus 0.911 to 0.936 for CNNs, but CS values are not comparable across architectures without qualification.Across 28 frozen configurations, every CAG was negative; the standard deviation of mean CS across seeds was at most 0.005.Source: §5.1 · p. 9, Table 3 · p. 10
  • Across 16 CNN configurations, accuracy and CS correlated at r = 0.586, while CS and ECE were not significantly related (r = −0.21, p = 0.43).Flip rates varied more by dataset than by operator; on CIFAR-10, JPEG flipped 21.7% of predictions versus 2.8% for brightness despite smaller input distance.Source: §5.1 · pp. 9–10

5.2. Error detection against confidence baselines

  • For frozen CNNs, CS trailed MSP by 0.057 AUROC (95% CI 0.043 to 0.073), and CS remained below MSP across model families.The Figure 2 comparison covers four model families; the frozen-CNN estimate comes from 48 runs.Source: §5.2 · pp. 10–11, Figure 2 · p. 11, Table 4 · p. 11
  • TTA mean confidence reached 0.939 against MSP at 0.920 on natively trained models; TTA agreement did not outperform MSP.The passage attributes the distinction to averaging predictions over views rather than measuring their disagreement.Source: §5.2 · p. 11
  • Among predictions with MSP ≥0.90, LS reached AUROC 0.508, while adding LS and CD to MSP and margin changed AUROC by −0.001.The partial correlation between margin and LS given MSP averaged 0.07 across configurations.Source: §5.2 · p. 11
  • DCF required seven forward passes per image and ran 7.9× to 15.2× slower than the baselines in mixed precision.Configuration rankings also depended on the operator set: Kendall’s τK averaged 0.44 with one operator and 0.88 with five.Source: §5.2 · p. 11

5.3. Reformulating CD and choosing α

Alternative definitions of confidence deviation and weight selection modestly changed consistency scores but did not bring them close to MSP. On clean data, displacement had little error-predictive value at a fixed margin, and the conformal evaluation tracked certification coverage and flips against its guarantee.

5.3. Reformulating CD and choosing α

  • The total-variation and rescaled CD variants reached AUROC 0.828 and 0.833 versus 0.819 for the original, but none came within 0.04 of MSP.Selecting α on calibration data matched α = 0.5 to three decimals; even test-set selection improved AUROC by less than 0.001.Source: §5.3 · p. 12
  • DCF variants separated stochastic-dropout heads from normal ones, but none detected an under-trained, under-confident head whose CS stayed near the fully trained range.The under-trained head had accuracy 0.793 and mean confidence 0.255, while its CS remained 0.937 to 0.945.Source: §5.3 · p. 12

5.4. Why consistency is redundant on clean data

  • The conformal evaluation reports certification coverage and realized P(certified ∧flip) against β, by dataset and model family, across 84 runs.Table 7 identifies the certificate guarantee as P(certified ∧flip) ≤β.Source: Table 7 · p. 13, Figure 3 · p. 13
  • At a fixed margin, negative mean displacement predicted errors at AUROC 0.515, and adding displacement and LS to a margin model changed AUROC by +0.003 on average.LS reached AUROC 0.535 within margin bins; the run-level test for displacement against chance gave p = 0.67.Source: §5.4 · pp. 12–13

5.5. The certificate and the bounds

The certificate held its stated guarantee across frozen runs, while calibrated per-sample bounds were more informative than the lemma-shaped bound. Figure 4 indicates that consistency’s advantage over the margin varies by corruption family.

5.5. The certificate and the bounds

  • Across 84 frozen runs, the conformal guarantee held at every β, and the largest realized P(certified ∧flip) was 0.003 at β = 0.05.At β = 0.05, certified images were correct more often than the evaluation set overall: 0.989 versus 0.826.Source: §5.5 · p. 13
  • Figure 4 shows that CS beat the margin only for noise corruptions; its AUROC difference and the gain from adding DCF features both grew with MSP over-confidence.The figure summarizes 14 natively trained models, with one point per corruption.Source: Figure 4 · p. 14
  • The DKW-corrected bound averaged 0.235 with Brier score 0.124 per image and was exceeded in none of 5,040 cells, though calibrated bounds remained about four to five times the observed flip rate.The lemma-shaped bound averaged 0.599 and was trivial in 55% of cells.Source: §5.5 · p. 14

5.6. Distribution shift: where consistency helps and where it does not

On CIFAR-10-C, pooled consistency scores remained slightly below confidence baselines, but consistency helped on noise corruptions where confidence was most overconfident. That gain largely failed to transfer to unseen corruption families.

5.6. Distribution shift: where consistency helps and where it does not

  • On CIFAR-10-C, CS reached AUROC 0.828, below MSP (0.835), DOCTOR (0.836) and negative entropy (0.837) across 19 corruptions and five severities.On self-generated CIFAR-10 corruptions, CS was 0.737 against MSP at 0.815; adding LS and CD to confidence-and-margin features left AUROC unchanged at 0.814.Source: §5.6 · p. 14
  • On four noise corruptions, CS beat the margin by 0.05–0.11 AUROC, and its gain across corruptions tracked MSP over-confidence (Spearman ρ = 0.88).The noise gain reached 0.13 when the combiner was fitted on the same corruption; blur alone scored 0.878 against 0.753 for the margin.Source: §5.6 · p. 14
  • With clean-only training, the gain on noise fell to +0.009; training on other corruption families yielded +0.002, not significant, so gains did not transfer to unseen shifts.A gain of +0.067 survived leave-one-corruption-out testing only when other noise corruptions were included in training.Source: §5.6 · p. 14
  • Increasing all six operators’ strength worsened CS AUROC from 0.903 at level 1 to 0.755 at level 5, while MSP remained 0.920; calibration selected level 1 in 40 of 42 runs.Source: §5.6 · pp. 14–15
  • For label-free accuracy estimation under self-generated shift, MSP thresholding had mean absolute error 0.049, versus 0.124 with CS replacing MSP and 0.091 for average confidence.Clean accuracy predicted held-out corruption-family accuracy with Spearman 0.99, while clean CS added nothing after controlling for accuracy (partial Spearman +0.08).Source: §5.6 · p. 15

5.7. Thresholds and regimes

CAG regime labels depend strongly on the descriptive tolerance δ, and clean CAG did not predict accuracy loss under shift. In adversarial tests, CS detected non-adaptive attacks but an adaptive attacker could evade detection at a substantial cost in attack success.

5.7. Thresholds and regimes

  • At δ = 0.02, 15 of 16 CNN configurations were consistency-dominant; at δ = 0.10, only the four OCTMNIST models were, and none were accuracy-dominant at any tested δ.Eleven of 16 configurations changed regime labels across the δ range 0.02 to 0.10; δ is a descriptive tolerance without operational interpretation because it was not independently validated.Source: §5.7 · p. 15
  • For shift alarms tested across corruption families, MSP reached balanced accuracy 0.94 and the margin 0.91, versus LS at 0.76 and CS at 0.72; clean CAG did not predict shift-related accuracy drop (Spearman 0.09, p = 0.78).A consistency floor would not bind because every Table 3 configuration had mean CS above 0.91.Source: §5.7 · p. 15

5.8. Adversarial and out-of-distribution inputs

  • Under non-adaptive PGD, CS separated clean from attacked images at AUROC 0.83, 0.80 and 0.73 for ε∞=2, 4 and 8/255, while MSP scored 0.370, 0.105 and 0.023.CS beat MSP and the margin in all 42 runs at every budget, but embedding displacement reached 0.89–0.94 and Mahalanobis overtook CS at 8/255 (0.88 against 0.73).Source: §5.8 · p. 15
  • With adaptive PGD at λ = 5, CS detection AUROC fell to 0.09 at 4/255 and 0.05 at 8/255, but only 6% and 9% of attacks still changed predictions, versus 98% and 100% for plain PGD.At every λ, CS ranked errors on attacked images with AUROC 0.79–0.87 while MSP scored 0.06–0.50; the captioned figure shows detection collapse alongside falling attack success.Source: §5.8 · pp. 15–16, Figure 5 · p. 16

5.9. Stable but wrong: OCTMNIST

The large negative OCTMNIST CAG persisted across stronger adaptation and in-domain training, so backbone-domain mismatch alone does not explain it. Prior correction recovered part of the gap, and induced label shift on CIFAR-10 reproduced the stable-but-wrong pattern.

5.9. Stable but wrong: OCTMNIST

  • CAG stayed below −0.10 in all 33 OCTMNIST runs spanning frozen CNNs, longer head training, full fine-tuning and training from scratch.Full fine-tuning improved CAG by 0.145 ± 0.077 for ResNet-18 and 0.158 ± 0.012 for MobileNetV2, but the gap remained −0.12 to −0.16.Source: §5.9 · p. 16
  • An OCTMNIST-only ResNet-18 had CAG −0.206; at matched accuracy 0.620, the in-domain model’s CAG was −0.331 versus −0.308 for the frozen ImageNet model.These comparisons do not support backbone-domain mismatch as the sole explanation, though each matched-accuracy comparison uses one point per trajectory.Source: §5.9 · p. 16
  • OCTMNIST’s skewed training prior contrasted with a balanced test split; drusen recall was 12% for frozen ResNet-18 and 33% after fine-tuning, while wrong drusen predictions were highly stable.On the validation split sharing the training prior, CAG nearly vanished for fine-tuned models (−0.021 and +0.002) and was smaller for frozen models (about −0.10 to −0.13).Source: §5.9 · p. 17, Table 10 · p. 17
  • Correcting to the true balanced test prior improved mean OCTMNIST CAG from −0.264 to −0.162, recovering about 40% of the gap; label-free EM correction improved it to −0.224.The true-prior correction improved 32 of 33 runs and raised drusen recall from 14% to 53%; the residual remained on the test split.Source: §5.9 · p. 17
  • With two CIFAR-10 classes reduced to 2% of training images, rare-class recognition was 9%, LS remained 0.82 and CAG fell from −0.060 to −0.175.Prior correction raised CAG to −0.083, while re-weighting the test set to the training prior raised it to −0.031; Table 11 describes the balanced test set and training-image fractions.Source: §5.9 · p. 17, Table 11 · p. 17

6. Discussion

On clean data and unseen shifts, consistency mostly re-expresses confidence, while it can add information when confidence is corrupted. CAG signals stable errors, often associated with prior mismatch, but its scope and interpretation are limited by the study’s settings and unvalidated thresholds.

6.1. What the results say about perturbation consistency

  • On clean data and unseen shifts, perturbation consistency was a costlier, noisier re-expression of confidence; at fixed margin, displacement carried little correctness information.The discussion recommends MSP, margin or entropy for ranking clean predictions by trustworthiness.Source: §6.1 · p. 17
  • When confidence is corrupted by noise or adversarial inputs, consistency carried information confidence lacked; the gain followed over-confidence, but consistency features helped only on observed, labelled shifts.The discussion identifies smoothing as a mechanism and prefers feature-space consistency over predicted-label consistency; it also points to adaptive-attacker limits.Source: §6.1 · p. 18, §6.3 · p. 18

6.2. What CAG measures

  • CAG equals mean instability minus error rate, so a large negative value indicates many wrong but stable predictions; class-prior mismatch is one common reason.CAG is best read as a prompt to check class priors and per-class recall, not as a reliability summary; it does not predict accuracy loss under shift, and regime labels depend on arbitrary δ.Source: §6.2 · p. 18

6.3. Scope and limitations

  • Most experiments used CIFAR-10-to-STL-10-scale images and mild operators; label preservation was screened automatically on three datasets, not validated by human raters.The study does not claim every perturbed image preserves its label.Source: §6.3 · p. 18
  • The adaptive-attack evidence uses one 20-step objective, and stronger or differently weighted attacks may break the observed trade-off; no adversarial-robustness claim is made.The conformal guarantee is marginal under exchangeability and is not formally covered on OCTMNIST because calibration and balanced test class priors differ.Source: §6.3 · p. 18
  • The corruption zoo analysis used self-generated rather than official CIFAR-10-C files, and the OCTMNIST gap not recovered by prior correction remains unexplained.The in-domain control was trained from scratch rather than using a retinal-pretrained backbone such as RETFound.Source: §6.3 · p. 18

7. Conclusion

The consistency score underperformed clean-output confidence on clean misclassification detection, but added information under noise and adversarial perturbations. The consistency-accuracy gap on OCTMNIST remained large and negative across training settings, with class-prior shift explaining part of it.

7. Conclusion

  • Across 201 evaluations, consistency was below MSP for clean misclassification detection in every model family, added no information when combined with MSP, and cost 8 to 15 times more compute.A label flip requires the clean logit margin to be at most the square root of two times the logit displacement; the resulting label-free conformal certificate held in every run.Source: §7 · p. 19
  • Consistency added information under noise corruptions and adversarial perturbations, but the gain did not transfer to unseen shift types.An adaptive attacker can trade attack success for stability.Source: §7 · p. 19
  • The consistency-accuracy gap was large and negative on OCTMNIST, and backbone mismatch alone did not explain it.The pattern persisted with longer head training, full fine-tuning, and training from scratch in-domain; at matched accuracy, the in-domain model was no better.Source: §7 · p. 19
  • Prior correction recovered about 40% of the gap, and inducing label shift on CIFAR-10 reproduced the pattern.The protocol provides a baseline-controlled way to evaluate future perturbation-based reliability scores.Source: §7 · p. 19

CRediT authorship contribution statement

The authors’ contribution statement assigns conceptualization and methodology to both authors, with distinct additional roles in implementation, analysis, supervision, validation, and writing.

CRediT authorship contribution statement

  • Amit Kalita is credited with software, investigation, formal analysis, visualization, and original-draft writing; Moirangthem Tiken Singh with supervision, validation, and review-and-editing.Both authors are credited with conceptualization and methodology.Source: CRediT authorship contribution statement · p. 19

Related papers

Loading 2610.36640v1…