Medical AI Encodes a "Feeling of Error": Verifying Cancer Segmentation via Internal Concepts

• The motivation is to flag high-risk cases for inspection or provide justifications clinicians can use to trust, refine, or override predictions.Source: §1 · pp. 1–2
• Output-level confidence or uncertainty cues offer little insight into failure causes, and greater detection sensitivity can reduce segmentation quality.The reported 20-point gain in failure-detection F1-score comes with an 8–10-point drop in segmentation DSC.Source: §1 · p. 2
• Failure cases activate fewer internal concepts with lower magnitudes than successful cases, forming a latent signature the authors call machine FOE.Source: §1 · p. 2, Figure 2 · p. 3
• The proposed two-stage framework extracts concepts across early, middle, and deep layers, then uses an interpretable classifier to detect failures and show concept contributions.Source: §1 · p. 3
• Internal concept representations deliver superior failure detection while providing intuitive explanations and maintaining high segmentation quality in prostate and pancreatic cancer experiments.Source: §1 · p. 3

• Concept activations form success- and failure-predictive groups, with deep layers showing stronger predictive power overall and middle layers more failure-predictive concepts.Source: Figure 3 · p. 4, §2 · p. 5
• A segmentation failure is a prediction whose quality metric score is at or below a predefined threshold τ.Source: §2 · p. 4
• Sparse Autoencoders decompose internal activations into sparse combinations of learned features, balancing reconstruction fidelity with sparsity during training.Source: §2 · p. 4
• Failure cases activate significantly fewer SAE concepts at lower magnitudes than successful cases (p < 10−5, Cohen’s d = 1.8).Source: §2 · p. 4
• SAE concepts align with clinical anatomy and lesions: concepts with IoU ≥0.5 receive annotation-based labels, and radiologists agreed with all presented labels.Source: §2 · p. 5
• Open questions include selecting failure-predictive concepts from dictionaries of 1K–10K entries and aggregating concepts across layers into a reliable detector.Source: §2 · p. 5

3.1 How to Capture All Potential Failure Signals?

• SAEs trained on early, middle, and deep ViT layers produce sparse concept vectors that are concatenated into a holistic failure representation.Six layers, L = {1, 3, 5, 7, 9, 11}, are sampled across the backbone; the representation preserves layer-specific concepts.Source: §3.1 · p. 6
• p < 1e−5 and d > 1.8: correct predictions and failures show statistically significant separation with minimal overlap in the internal failure representation.The authors take this as evidence that internal concepts encode discriminative features suitable for failure detection.Source: §3.1 · pp. 6–7

3.2 How to Identify Predictive Patterns of Failure?

• A supervised classifier maps failure representations to failure predictions and feature-importance scores, with XGBoost used to capture nonlinear interactions across concepts and layers.Training labels distinguish correct predictions from failures, based on ground-truth annotations.Source: §3.2 · p. 7

3.3 Implementation and Discussion

• SAEs operate directly on patch-level embeddings to preserve spatial granularity, enabling concept activation maps to show which image regions activate each concept.The method applies each layer's encoder to spatial patch embeddings and extracts a 2D activation map for a selected concept.Source: §3.3 · p. 8
• Patch-level failure scores support selective mask correction: retain a predicted patch when its local failure score is below threshold η.The authors say this preserves high-confidence true-positive regions that output-based methods would discard when flagging a whole prediction as failed.Source: §3.3 · p. 8
• Unlike concept bottleneck models, the framework is post-hoc and uses unsupervised SAE-discovered concepts without manual concept annotations; some predictive concepts are non-semantic.The authors note that these concepts need not be fully aligned with human language.Source: §3.3 · p. 8

4.1 Data, Baselines, Metrics, and Hyperparameters

• The evaluation uses PI-CAI, Prostate158, and PanTS, compares post-hoc output-based uncertainty baselines, and reports DSC, F1, AUROC, and FPR95.The study targets post-hoc deployment with frozen pretrained segmentation systems; its backbone is MedSAM and the SAE settings are dictionary size 1,536 and sparsity 8.Source: §4.1 · pp. 8–9

4.3 Failure Detection

• Internal concepts break the baseline Pareto tradeoff, achieving high failure-detection F1 and segmentation DSC simultaneously.The comparison covers PI-CAI and PanTS.Source: Figure 5 · p. 10, §4.3 · p. 10

4.2 Failure Interpretation

• SAE concepts align with anatomy; failures activate tumor concepts weakly or not at all, whereas successful cases produce strong tumor-concept activations.Prostate concepts include the peripheral zone, gland, and tumor, with similar anatomical alignment reported for pancreatic cancer.Source: §4.2 · pp. 9–10

4.3 Failure Detection

• On unseen Prostate158, concept-based detection outperforms confidence-based baselines across metrics, and success and failure cases remain separable in concept space.Detectors were trained on PI-CAI without fine-tuning on Prostate158.Source: Table 1 · p. 11, §4.3 · p. 11, Figure 6 · p. 11
• The method identifies hallucinated lesions and boundary errors, while Fig. 7 illustrates false-positive removal that improves segmentation in MRI and CT examples.The classifier’s most important concepts are tumor-related, alongside anatomical boundary concepts that capture gland edges and extra-prostatic regions.Source: §4.3 · p. 11, Figure 7 · p. 12, Figure 8 · p. 12

4.4 Ablation Study

• SAE architecture ablations select sparsity L0 = 8 and dictionary size 1,536; across layer depths, deep concepts achieve F1=57.6 and mid-layer concepts achieve DSC=48.8.Combining early, middle, and deep concepts achieves the best DSC and F1; XGBoost gives the best results among tested classifiers.Source: §4.4 · pp. 12–13, Table 3 · p. 13, Table 4 · p. 13
• Routine deployment adds +3.7% overhead; the decoder’s approximately 3.4 ms overhead is invoked only at intervention time.The full failure-detection pipeline runs at around 5.66 img/s.Source: §4.4 · p. 13, Table 5 · p. 13

Medical image segmentation for cancer detection

• Cancer segmentation has advanced across modalities, but clinically significant false positives and false negatives remain possible.False positives may lead to unnecessary biopsies, while false negatives can miss critical lesions.Source: §5 · pp. 13–14

Failure detection in medical AI

• Output uncertainty signals can flag potential errors but provide no insight into failure causes and may generalize poorly across datasets.Image-quality assessment and ensemble disagreement also lack interpretability.Source: §5 · p. 14

Interpretability methods

• Unlike post-hoc explanation or manually annotated concept methods, this work uses unsupervised SAE concepts for actionable failure detection in medical segmentation.Related approaches include attribution methods, concept-based models, and unsupervised concept discovery targeting natural images.Source: §5 · p. 14

Conclusion

• Concept activation patterns distinguish true from false positives in prostate and pancreatic segmentation, supporting failure detection and interpretable correction.The conclusion also reports robust zero-shot generalization across datasets where confidence-based methods collapse.Source: §6 · p. 14

Limitations and future work

• The current method captures concept occurrence rather than causal relationships, and SAEs are trained separately for each cancer type.The authors also identify more specific failure-mode classification as future work.Source: §6 · p. 14
• Extending the framework to missing-modality segmentation is proposed as future work, including concepts for identifying missing information and guiding adaptive fusion or imputation.Source: §6 · p. 15

• The supplementary material is titled “Cancer Segmentation via Internal Concepts.”Source: Supplementary Material for Medical AI… · p. 20

A Segmentation Failure Detection for Brain Tumor

• The MSD-Brain dataset contains 484 MRI scans, split into 310 training, 78 validation, and 96 testing scans, with three tumor-component segmentation tasks.The components are the complete tumor, tumor core, and enhancing portion.Source: A Segmentation Failure Detection for… · p. 20
• On MSD-Brain, the method surpasses the baseline Pareto front with both higher DSC and higher F1.Each marker is a baseline method evaluated across thresholds [0.5, 0.8].Source: Figure 9 · p. 20
• Brain-tumor concepts localize to tumor characteristics, and examples show detected failures followed by corrected masks.The concepts include enhancing regions, necrotic core, edema, and tumor margins; the reported findings align with prostate and pancreatic tumors.Source: Figure 10 · p. 20, A Segmentation Failure Detection for… · p. 21

B Model Details

• The model replaces MedSAM’s manually prompted encoder with an automated overload encoder that generates surrogate prompts directly from images.ResNet is given as an example of an overload encoder.Source: B Model Details · p. 21
• MedSAM’s mask decoder is modified to output multiple classes for structures such as glands, zones, and lesions.Source: B Model Details · p. 21
• The starting model is pretrained MedSAM, whose usual prompt encoder requires manual input such as a bounding box.Source: B Model Details · p. 21

C Additional Discussion

• Layer 9 concepts have ∆random ≈0.10−0.13; Layer 5 is particularly effective for false positives, while deeper layers better detect true positives.Layers 1 and 5 provide complementary information, supporting integration across layers.Source: C Additional Discussion · pp. 21–22, Figure 11 · p. 21
• Amplifying tumor-related latent #469 sharply reduces DSC, while two controlled experiments and SAE reconstruction alone slightly improve it.The intervention provides evidence that #469 influences model output; the authors suggest sparse reconstruction may also act as a denoiser.Source: C Additional Discussion · p. 22, Figure 12 · p. 22, Table 6 · p. 22
• The SAE detector outperforms test-time augmentation in F1, accuracy, and DSC after failure correction, using one model pass plus SAE encoding rather than six full passes.Reducing TTA from K=6 to K=4 yields nearly identical F1 (0.591).Source: C Additional Discussion · p. 22, Table 7 · p. 22
• The brain-tumor example traces false-positive correction from concept-localized prediction regions to detected false positives and a final mask after removal.Source: Figure 13 · p. 23