Statistical Testing for Multiple Instance Learning via Selective Inference with Applications to Computational Pathology

1 Introduction

• High-attention patches are used as visual explanations in computational pathology, but attention scores may reflect bias or optimization artifacts rather than clinically meaningful evidence.The paper therefore identifies a need for statistical evidence about whether selected instances differ from normal instances.Source: §1 · p. 2
• The study formulates evaluation of high-attention instances as hypothesis testing against a similar, adaptively selected normal reference because normal instances can be heterogeneous.The comparison uses a distance-based statistical test.Source: §1 · p. 2
• Because both target and reference instances are selected from observed data, ignoring those selection steps can invalidate standard hypothesis tests; the proposed SI framework conditions on the selection events to derive valid p-values.The framework addresses attention-based target selection and adaptive reference selection.Source: §1 · p. 2
• Experiments show Type-I error control on synthetic and MNIST-based data and higher statistical power than conventional over-conditioning, with practical applicability to pathological WSIs.The paper also reports a higher rejection proportion than over-conditioning on tumor WSIs in CAMELYON16.Source: §1 · pp. 2–3
• The paper presents its SI framework as the first attempt to quantify the statistical significance of MIL-selected instances and notes that the principle may extend beyond ABMIL when selection events can be characterized.ABMIL is the proof-of-concept, chosen for its explicit attention mechanism.Source: §1 · pp. 2–3

2.1 MIL-based classification problem

• In MIL, class labels apply to bags, while the instances within each bag remain unlabeled.For pathology, each WSI is a bag and its image patches are instances; cancerous patches are positive and normal patches negative.Source: §2.1 · pp. 3–4

2.2 ABMIL and attention-based instance selection

• ABMIL extracts instance features, computes attention logits, and aggregates features into a bag-level representation for classification.The model is trained end-to-end to minimize error between predicted and observed bag labels.Source: §2.2 · pp. 4–5
• High-attention instances are natural targets for testing whether they differ from a representative reference, but adaptive selection of both instances makes standard testing invalid.Attention scores alone do not establish statistical evidence that selected instances differ from normal instances.Source: §2.2 · p. 5

3 Statistical testing formulation for high-attention instances

• The test instance is modeled as a fixed signal plus isotropic Gaussian noise, with arbitrary fixed signal vectors and known noise variance for theoretical analysis.The Gaussian assumption applies to perturbations, not to the distribution of underlying signals.Source: §3 · p. 6
• For each selected high-attention instance, the test asks whether it significantly differs from a representative normal reference instance.The reference is the medoid among the k nearest normal instances in feature space, providing a local comparison amid normal-tissue heterogeneity.Source: §3 · p. 6
• The test statistic is the Euclidean distance between the selected instances, standardized to remove dependence on the noise scale.Under the null with a reference fixed independently of the data, the statistic has a chi distribution with d degrees of freedom.Source: §3 · p. 7
• The fixed-reference naive p-value does not generally control nominal Type-I error because both the target and reference are selected using observed data.The proposed selective-inference framework characterizes those selection events to compute valid conditional p-values.Source: §3 · p. 7

4.1 Vector-form test statistic

• In vector form, the test statistic uses a projection matrix whose nonzero blocks correspond to the test instance and selected representative reference.The matrix is an orthogonal projection, satisfying P^⊤=P and P^2=P.Source: §4.1 · p. 8

4.2 Naive p-value

• The naive p-value uses a chi null distribution that assumes the target and reference were fixed independently of the observed data, so it does not guarantee nominal Type-I error control after adaptive selection.The fixed-selection null distribution is valid only when those instances were prespecified.Source: §4.2 · p. 8

4.3 Selective p-value

• Selective inference conditions on the observed attention and reference-selection outcomes, then computes an upper-tail probability from the resulting truncated chi distribution.A nuisance statistic fixes the projected direction and orthogonal component, leaving the standardized statistic as the one-dimensional quantity varying over the feasible region.Source: §4.3 · p. 9
• The selective p-value exactly controls conditional Type-I error at every significance level under the null hypothesis.The conditional null distribution is a truncated chi distribution with d degrees of freedom.Source: §4.3 · p. 9

4.4 Selection maps and selection events

• The selection event preserves the high-attention outcome and final medoid reference, but does not require the intermediate k-nearest-neighbor set to stay fixed.The threshold is fixed during inference; the feature space for retrieval and medoid selection is chosen before inference.Source: §4.4 · p. 10

5.1 Experimental settings

• The evaluation compares Type-I error and power where ground truth is available, using ablations that omit either attention-based selection or adaptive reference selection, plus OC and Bonferroni.For CAMELYON16, normal- and tumor-WSI rejection proportions serve as operational proxies; the study uses one center and macro-metastatic tumor cases.Source: §5.1 · p. 11, §5.4 · pp. 14–15

5.2 Synthetic experiments

• Across the tested synthetic settings, the proposed method controlled Type-I error at the nominal level and achieved higher statistical power than OC.Ablation 2 was conservative, while Bonferroni was highly conservative; the naive method also showed conservative behavior in these settings.Source: §5.2 · p. 12, Figure 3 · p. 13
• Under strong attention-based selection, the proposed method stayed near the nominal Type-I error rate, while the naive method and Ablation 1 showed substantial inflation.The stress test used a threshold at the 99.99th percentile of logits from independent normal instances and varied Mref = k over 1, 3, 5, and 10.Source: §5.2 · p. 12, Figure 4 · p. 13

5.3 MNIST-based experiments

• Across positive digit classes in both three- and five-cluster MNIST settings, the proposed method controlled Type-I error and had higher power than OC.Ablation 2 was conservative, and Bonferroni was overly conservative.Source: §5.3 · p. 14, Figure 5 · p. 17

5.4 Real-world experiments on CAMELYON16

• On CAMELYON16, the proposed method’s rejection proportion was close to nominal on normal WSIs and substantially higher than OC on tumor WSIs.The authors describe these rejection proportions as operational proxies because the underlying signal equality for patch pairs is not directly observed.Source: §5.4 · p. 15, Table 1 · p. 16

6 Conclusion

• The framework conditions on attention-based target selection and the final representative reference, not the intermediate k-nearest-neighbor set, and uses parametric programming to characterize the full compatible truncation region.This procedure yields valid selective p-values.Source: §6 · p. 19
• Synthetic and MNIST-based experiments showed Type-I error control and higher power than over-conditioning, while CAMELYON16 showed a higher tumor-WSI rejection proportion than over-conditioning.The authors characterize the framework as a statistically principled way to assess attention-selected instances against adaptively selected references.Source: §6 · p. 19
• The evaluation demonstrates the framework using ABMIL as a proof-of-concept; broader application depends on characterizing selection along a one-dimensional conditional path.The authors identify extending the approach to modern MIL and foundation-model pathology architectures as future work.Source: §6 · p. 19

A Summary of notation; A.1 Multiple instance learning; A.2 Test and reference instances

• The notation distinguishes bags and instances, model and attention quantities, and test and reference samples and their signal-plus-noise components.It also defines the test statistic, selective and naive p-values, and maps for attention, neighbor retrieval, representative-reference selection, and their combined outcome.Source: A Summary of notation · p. 24, §A.1 · p. 24, §A.2 · p. 24

A.3 Test statistics and selective inference

• The test statistic is the norm of the projected random vector, standardized by noise scale; the projection represents the contrast between the test instance and selected medoid reference.The notation also names the nuisance statistic and the multiplicity factor for attention-based selection.Source: §A.3 · p. 24

A.4 One-dimensional conditional path and selection events

• The one-dimensional conditional path preserves selection outcomes over a feasible region, with the overall truncation region formed by intersecting the attention and representative-reference regions.The notation distinguishes the local region where the neighbor set is fixed from the region where the medoid remains selected.Source: §A.4 · p. 25

• The theoretical guarantees assume that the true noise variance is known.Source: B Proofs of theorems · p. 25
• Synthetic and MNIST-based experiments use the true data-generation variance for inference.Source: B Proofs of theorems · p. 25
• CAMELYON16 substitutes an estimate from independent normal instances, and the proofs do not establish exact finite-sample validity for this plug-in implementation.Source: B Proofs of theorems · p. 25

B.1 Proof of Theorem 4.1

• The proof fixes the observed selection outcome and its deterministic projection matrix, with E=(A, M) and the intermediate k-nearest-neighbor set excluded from the conditioning event.Source: §B.1 · p. 25
• Before conditioning, the projected direction and the orthogonal component yield independence between e^T(Y) and Q(Y).Source: §B.1 · p. 26
• Conditioning on the observed selection event restricts e^T(Y)'s support to Z, giving a chi distribution with d degrees of freedom truncated to Z.Source: §B.1 · p. 26

B.2 Proof of Theorem 4.2

• The selective p-value has conditional rejection probability α for any α∈(0, 1), yielding exact Type-I error control for each selection outcome whose null holds.Source: §B.2 · p. 26

C Selection event characterization

• The overall selection map combines attention-based target selection A with adaptive reference selection M, which retrieves k nearest instances and chooses their medoid.Source: C Selection event characterization · p. 26

C.1 Attention selection event

• The attention-selection event preserves whether the test instance is selected by its pre-sigmoid attention logit, using a threshold fixed before selective inference.Source: §C.1 · p. 27

C.2 k-nearest-neighbor partition

• The k-nearest-neighbor set partitions the conditional path into local regions but is not itself conditioned upon.Source: §C.2 · p. 28

C.3 Medoid selection event

• Within each fixed neighbor set, the medoid minimizes summed squared distances; the same final medoid can be retained across different neighbor sets.Source: §C.3 · pp. 28–29

C.4 Overall selection event

• The full selection event simultaneously preserves the attention outcome and selected reference, while permitting the k-nearest-neighbor set to change.Source: §C.4 · p. 29

C.5 Computation of the truncation region

• The truncation region can contain disjoint intervals, found by exploring local regions and combining intervals that preserve both observed selections.Unlike over-conditioning, the procedure retains the entire region consistent with the observed selection outcome, not only the local interval containing the observed statistic.Source: §C.4 · p. 29, §C.5 · p. 29

D Experimental details and additional results

• The appendix adds experimental details omitted from the main paper and supplementary results across parameter settings.Source: D Experimental details and additional… · p. 30
• Experiments ran in Python on an x86_64 Linux computer with two 20-core Intel Xeon Gold 6230 CPUs, except UNI2 feature extraction.Source: D Experimental details and additional… · p. 30
• CAMELYON16 UNI2 feature extraction ran on a separate Linux computer equipped with an NVIDIA RTX A6000 GPU.Source: D Experimental details and additional… · p. 30

D.1 Synthetic experiments

• Each dimension-and-noise setting used independently trained ABMIL models and newly generated bags, with 1,000 positive and 1,000 negative 10-instance bags per epoch.Positive bags contained one to ten positive instances; negative bags contained only negative instances.Source: §D.1 · p. 30
• The synthetic ABMIL model used an MLP feature extractor, attention network, and bag classifier, then trained for 10 epochs with binary cross-entropy and Adam.Attention weights used sigmoid followed by within-bag softmax; only the feature extractor and attention network were used for subsequent selective inference.Source: §D.1 · p. 30
• Power rose with input dimension and fell with noise variance for both the proposed method and OC.The proposed method's power advantage over OC grew with input dimension.Source: §D.1 · p. 30
• Across dimensions, noise variances, and k values, the proposed method kept Type-I error near nominal and consistently exceeded OC in power.OC also controlled Type-I error, while Bonferroni was highly conservative; Ablation 1 was near nominal in examined settings and Ablation 2 was conservative.Source: §D.1 · p. 30, Figure 7 · p. 31

D.2 MNIST-based experiments

• The experiments used digit 0 as the negative class and one digit from 1–9 as the positive class, with noisy Gaussian clusters centered on MNIST images.Images were downsampled to 14 × 14, flattened to 196-dimensional vectors, and normalized to [0, 1]; training and inference used separate cluster centers.Source: §D.2 · p. 31
• Training used 1,000 positive and 1,000 negative bags per epoch, each with 10 instances, and regenerated Gaussian noise each epoch.Positive bags contained one to ten positive-class instances; negative bags contained only negative-class instances.Source: §D.2 · p. 32
• The MNIST ABMIL model mapped 196-dimensional inputs to 32-dimensional features and trained for 10 epochs with binary cross-entropy and Adam.Only the feature extractor and attention network were retained for subsequent selective inference.Source: §D.2 · p. 32
• Across three- and five-cluster settings and examined k values, the proposed method kept Type-I error near nominal and achieved higher power than OC.Bonferroni was substantially conservative and markedly less powerful than both methods; the results indicate robustness to cluster count and k.Source: §D.2 · p. 32, Figure 8 · p. 33, Figure 9 · p. 34

D.3 Real-world experiments on CAMELYON16

• For training, WSIs were divided into 50-patch bags, with up to 50 bags per slide and bag assignments regenerated each epoch.The model used pooled 192-dimensional UNI2 features mapped to a 32-dimensional representation, with an attention network and bag-level classifier.Source: §D.3 · p. 35
• The trained ABMIL model achieved 0.897 slide-level classification accuracy on the test set.All patches from each WSI were treated as a single bag for slide-level evaluation.Source: §D.3 · p. 35
• Across the remaining six test-set tumor WSIs, the proposed method generally identified more statistically significant regions than OC, while Bonferroni yielded substantially fewer rejections.Figures 10 and 11 show ground-truth masks alongside p-value maps for the proposed method, OC, and Bonferroni; red and green boxes mark selected instances with p < 0.05 and p ≥ 0.05, respectively.Source: §D.3 · p. 35, Figure 10 · p. 36, Figure 11 · p. 37