Source-linked AI summary
Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
Wentao Zhu, Qi Lou, Yeeleng Scott Vang, Xiaohui Xie
TL;DR
Mammogram classification traditionally requires costly detection or segmentation annotations, while automated analysis must address false positives and clinician variability. The paper proposes end-to-end deep multi-instance networks that classify whole mammograms from patches without such annotations, using max pooling, label assignment, and sparsity schemes. On INbreast, the networks demonstrate more robust performance than previous work using detection or segmentation annotations, with limitations including fixed label-assignment parameterization and future need for larger datasets and multi-scale modeling.
Problem
Traditional mammogram classification requires costly bounding-box or segmentation annotations, motivating classification from raw whole mammograms.
Method
The paper uses end-to-end deep multi-instance networks that treat mammogram patches as instances and explore max pooling, label assignment, and sparsity schemes.
Results
The proposed networks demonstrate more robust whole-mammogram classification performance on INbreast than previous work using detection or segmentation annotations in training.
Takeaways & Limitations
The networks learn whole-mammogram and malignant-patch predictions without explicit bounding-box or segmentation ground-truth annotations.
Takeaways & Limitations
The label assignment scheme can misclassify small masses because it uses a constant k across mammograms, while future work targets multi-scale modeling, adaptive k, and larger datasets.
Abstract
from arXiv · showhide
Mammogram classification is directly related to computer-aided diagnosis of breast cancer. Traditional methods requires great effort to annotate the training data by costly manual labeling and specialized computational models to detect these annotations during test. Inspired by the success of using deep convolutional features for natural image analysis and multi-instance learning for labeling a set of instances/patches, we propose end-to-end trained deep multi-instance networks for mass classification based on whole mammogram without the aforementioned costly need to annotate the training data. We explore three different schemes to construct deep multi-instance networks for whole mammogram classification. Experimental results on the INbreast dataset demonstrate the robustness of proposed deep networks compared to previous work using segmentation and detection annotations in the training.
1 Introduction
Whole-mammogram classification is motivated by costly annotation requirements and the need for automated tools that improve detection while reducing false positives. The proposed deep multi-instance approach treats mammogram patches as instances and explores three learning schemes, motivated by the sparsity of masses.
- Breast cancer screening can produce high false positives, clinician variability, and over-diagnosis, motivating robust automated mammographic analysis.
- Traditional classification methods require costly bounding-box or segmentation annotations and may rely on hand-crafted features with limited transferability.
- A whole mammogram is modeled as a bag of patches, enabling classification from raw, un-annotated images through deep multi-instance learning.
- The framework explores max pooling, label assignment, and sparsity schemes for whole-mammogram classification.
- The proposed networks are presented as end-to-end methods intended to avoid detection or segmentation annotations during training.
- Masses average 329 × 325 pixels within mammograms averaging 1,474 × 3,086 pixels, occupying about 2% of the whole image.
2 Related Work
Prior mammogram classification and deep multi-instance methods commonly depend on annotated regions, segmentation ground truth, or max pooling that uses only the highest-probability patch. This paper explores task-related deep multi-instance models for whole-mammogram classification.
- Earlier mammogram methods used hand-crafted or deep-network approaches that required annotated mass regions, detection ROIs, or segmentation boundaries.
- Multi-instance learning research addressed instance-label ambiguity, support-vector classification, marginalized kernels, and multi-class extensions.
- Deep networks have been combined with multi-instance learning for natural-image auto-annotation, sentiment analysis, discriminative patch discovery, and patch-based prediction fusion.
- Existing deep multi-instance approaches commonly use max pooling, considering only the patch with the maximum probability.
3 Deep Multi-instance Networks for Whole Mammogram Mass Classification
The paper builds end-to-end deep multi-instance networks that classify whole mammograms from unannotated patches, using max pooling, label assignment, and sparsity schemes. The designs progressively incorporate multiple patch predictions and the prior that malignant masses occupy a small mammogram region.
- Common network: A CNN extracts features for all mammogram patches simultaneously, then shared logistic regression and sigmoid layers produce malignant probabilities for each patch.Feature-map positions correspond to patches, and shared weights are applied across positions.
- Max pooling-based multi-instance learning: Max pooling classifies a mammogram using the largest patch-level malignant probability, matching the assumption that one malignant mass makes the bag positive.Negative mammograms should have all patch probabilities near zero, whereas positive mammograms should have at least one near one.
- Label assignment-based multi-instance learning: Label assignment assigns the whole-mammogram label to the first k ranked patches and negative labels to the remaining patches.This explores all patches during training and behaves like data augmentation, but k is selected by cross-validation and is difficult to estimate.
- Sparse multi-instance learning: The sparse scheme uses the small mass-region prior to encourage patch probabilities toward zero while retaining the top-ranked patch as indicative of the mammogram label.Masses occupy about 2% of the whole mammogram on average, motivating an L1 sparsity constraint.
- Sparse multi-instance learning: Sparse multi-instance learning avoids assigning labels to every patch and instead models the overall statistical property of ranked patch probabilities.It is presented as an L1-constrained alternative to the exact k-sparse label-assignment formulation.
4 Experiments
The experiments evaluate end-to-end deep multi-instance networks on INbreast using five-fold validation and compare three multi-instance schemes with related annotated approaches. Sparse multi-instance learning achieves the best AUC, while the proposed networks improve over baseline models and can exploit patch-level information without detection or segmentation annotations.
- Experimental setup: The models are evaluated on INbreast with five-fold cross-validation, using three folds for training and one for validation in each test fold.The dataset contains 410 mammograms, including 94 with malignant masses.
- Experimental setup: The comparison includes manually or semi-automatically annotated methods, pretrained AlexNet variants, and the proposed max pooling, label assignment, and sparse multi-instance networks.The proposed methods operate on whole mammograms represented as collections of patches.
- Accuracy comparison: 2%, 3%, and 2% improvements are reported for pretrained AlexNet, pretrained AlexNet+Max Pooling MIL, and pretrained AlexNet+Label Assign. MIL, respectively, over corresponding non-pretrained models.The authors attribute these gains to the usefulness of ImageNet features for mammogram-related networks.
- ROC and AUC results: Sparse deep multi-instance learning provides the best AUC, followed by label assignment-based deep multi-instance learning, and the deep multi-instance networks improve greatly over baseline AlexNet models.AUC is averaged across the five test folds because malignant mammograms comprise only 20% of the dataset.
- Patch-level analysis: The visualizations show that the networks predict both whole-mammogram malignancy and malignant probabilities for individual patches without explicit bounding-box or segmentation ground truth.This supports learning mass regions from unannotated whole mammograms.
- Interpretation: The models train on all patches, reducing the possibility of overlooking malignant patches compared with approaches that first detect or segment regions.The reported advantage concerns patch utilization during end-to-end training.
5 Discussions
The discussion shows that deep multi-instance networks can identify malignant patches and mass regions while classifying whole mammograms without explicit localization annotations. It also identifies distinct errors associated with max pooling and fixed-k label assignment.
- Patch-level interpretation: The networks learn both whole-mammogram predictions and malignant-patch predictions, allowing mass regions to be learned without explicit bounding-box or segmentation annotations.The patch responses are visualized as malignant probabilities.
- Observed errors: Max pooling misses some malignant patches because training considers only the patch with maximum malignant probability.The reported misses occur in mammograms (a), (c), and (d).
- Observed errors: Label assignment misclassifies some patches in mammogram (d), possibly because a constant k can misclassify small masses.The explanation is presented as a possible reason rather than a demonstrated mechanism.
- Potential application: The authors identify automatic weak mass annotation as a potential application for computer-aided diagnosis.This application follows from the learned patch-level malignant probabilities.
6 Conclusion
The paper presents end-to-end deep multi-instance networks for whole-mammogram classification without detection or segmentation annotations, reporting more robust performance than previous work. It identifies sparse multi-instance learning and several extensions for future improvement and broader application.
- End-to-end deep multi-instance networks classify whole mammograms directly without detection or segmentation annotations during training.
- The proposed networks use ranking-based label assignment and sparse multi-instance learning to address the sparsity of mammographic masses.
- Future work includes multi-scale modeling, adaptive estimation of k, and using deep multi-instance learning to identify potentially malignant patches.
- The method is described as applicable to bio-image problems requiring expert knowledge or manual labeling, especially when regions of interest are small or sparse.