PaperlayerView exact revision on arXiv

Source-linked AI summary

Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions

Li Yao, Jordan Prosky, Eric Poblenz, Ben Covington, Kevin Lyman

arXiv:1803.07703v1cs.CV

TL;DR

Chest X-ray abnormalities vary in appearance and scale, while training data often lacks costly segmentation labels. The paper combines multi-resolution learning with weakly supervised localization and LSE-LBA pooling. It reports state-of-the-art performance on 9 of 14 abnormalities and higher-resolution saliency maps.

  • Problem

    Chest X-ray findings vary in scale, and segmentation labels are often unavailable or costly to collect despite the value of localization.

    Source: §1.1 · p. 1, §1.2 · p. 2

  • Method

    The model combines multi-resolution learning and multi-instance localization with LSE-LBA pooling, which imposes a learnable lower-bounded sharpness prior.

    Source: §1.4 · p. 3, §3.5 · p. 10, §3.6 · p. 10

  • Results

    The model outperforms the state of the art on 9 of 14 abnormalities, is on par on 3 others without extra data, and generates higher-resolution saliency maps.

    Source: §4.3 · p. 13, §5 · p. 15

  • Takeaways & Limitations

    Pathology-based saliency maps can provide visual attention and transparency for clinical workflow and model-bias inspection.

    Source: §1.2 · p. 2

Abstract

from arXiv · show

Diagnostic imaging often requires the simultaneous identification of a multitude of findings of varied size and appearance. Beyond global indication of said findings, the prediction and display of localization information improves trust in and understanding of results when augmenting clinical workflow. Medical training data rarely includes more than global image-level labels as segmentations are time-consuming and expensive to collect. We introduce an approach to managing these practical constraints by applying a novel architecture which learns at multiple resolutions while generating saliency maps with weak supervision. Further, we parameterize the Log-Sum-Exp pooling function with a learnable lower-bounded adaptation (LSE-LBA) to build in a sharpness prior and better handle localizing abnormalities of different sizes using only image-level labels. Applying this approach to interpreting chest x-rays, we set the state of the art on 9 abnormalities in the NIH's CXR14 dataset while generating saliency maps with the highest resolution to date.

1 Introduction

Chest X-ray abnormalities vary in scale, and clinicians benefit from both global predictions and localization, but medical training data rarely includes costly segmentation labels. The paper motivates multi-resolution, weakly supervised localization using image-level labels.

1.1 The Challenge of Diagnosing Abnormalities at Different Scales

  • Abnormalities range from globally visible cardiac enlargement to subtle nodules and patterns needing both macro- and micro-level inspection.Radiologists’ zooming workflow supports analyzing images at multiple resolutions.Source: §1.1 · pp. 1–2

1.2 The Challenge of Obtaining Medical Interpretation Beyond Global Predictions

  • Localization can provide visual attention and transparency, but segmentation labels are costly or unavailable, motivating localization from global labels.The paper notes that saliency maps can also help expose model biases.Source: §1.2 · p. 2

1.3 Weakly Supervised Multi-Instance Learning

  • The paper frames image localization as MIL, treating an image as a bag of instances whose labels can be inferred from the bag-level label.The cited common MIL assumption labels a bag positive when it contains one or more positive instances.Source: §1.3 · p. 2

1.4 Main Contributions

  • LSE-LBA adds a sharpness prior to multi-resolution MIL for image-label-only localization, refining lower-resolution cues into higher-resolution saliency maps.Figure 1’s visualization uses four models with increasing target resolutions.Source: §1.4 · p. 3, Figure 1 · p. 3

2 Related work

Prior work combines features across resolutions and uses weak supervision for localization, but medical approaches have often relied on one resolution, pixel-level labels, or ad-hoc saliency methods. This paper situates its architecture among those approaches.

2.1 Multiple Resolution Methods in Computer Vision

  • Multi-resolution methods combine and refine upsampled feature maps, while this model replaces pixel-wise supervision with image-wise MIL.The model uses nearest-neighbor upsampling followed by learnable nonlinear refinement.Source: §2.1 · p. 4

2.2 Weakly Supervised Learning

  • Weakly supervised localization uses image-level labels and MIL pooling, while CAM methods generate localization cues ad hoc from classification networks.The paper distinguishes its explicitly trained localization from CAM’s post-hoc cues.Source: §2.2 · pp. 4–5

2.3 Medical Deep Learning

  • In chest X-rays, prior localization approaches used single-resolution outputs, pre-trained networks, or class activation maps; this model fuses features across resolutions.Its architecture uses ResNet for resolution reduction, DenseNet per resolution, and fusion before pooling.Source: §2.3 · p. 5, Figure 2 · p. 6

3 Proposed Model

The model combines multi-resolution feature processing with LSE-LBA pooling to produce saliency maps and image-level predictions using image-level labels. Its design progressively refines features from coarse to fine resolutions.

3 Proposed Model

  • The model produces 64 × 64 saliency maps, compared with the prior highest resolution of 32 × 32, while extending U-net-like propagation to weak supervision.The model uses only image-level annotations rather than pixel-wise labels.Source: §3 · p. 7, §3.6 · p. 10

3.1 Reducing Resolutions with Residual Connections

  • Residual connections support resolution reduction, typically halving spatial resolution and doubling channels while retaining an identity-like shortcut for easier optimization.Downsampling uses stride-2 convolutions; the nonlinear residual transformation is handled by g.Source: §3.1 · p. 7

3.2 Refining Resolutions with Dense Connections

  • Repeated residual propagation can let the network ignore nonlinear capacity, so dense connections transform all previous feature maps while retaining direct connections that ease optimization.The features are concatenated channel-wise before resolution-preserving nonlinear transformation.Source: §3.2 · pp. 7–8

3.3 Combining Resolutions with Upsampling and Multi-Scale Dense Connections

  • Coarse-to-fine fusion upsamples lower-resolution features, concatenates them with features at the current resolution, and applies nonlinear refinement.Nearest-neighbor upsampling was selected because the authors observed no significant performance difference among the considered choices.Source: §3.3 · p. 8

3.4 The Choice of Pooling Function for Multi-instance Learning

  • Max and average pooling can respectively underestimate and overestimate object size; Noisy-OR and generalized mean offer alternatives, but can face numerical underflow.The generalized-mean parameter controls behavior between max-like and average-like pooling, while Log-Sum-Exp is described as similar to generalized mean and numerically resilient.Source: §3.4 · p. 9

3.5 Log-Sum-Exp Pooling with Lower-bounded Adaptation (LSE-LBA)

  • LSE-LBA balances average and max pooling while avoiding underflow when saliency values are near zero; its learned adaptation is lower-bounded to encode a sharpness prior.A larger lower bound encourages less diffuse saliency modes, and sigmoid activations keep saliency values within [0, 1] so pooled scores remain probabilities.Source: §3.5 · pp. 9–10

3.6 Weakly Supervised MIL Cost Function

  • The highest-resolution fused feature map is treated as a bag of instances, whose class probabilities form a saliency map pooled into image-level probabilities for cross-entropy training.A shared class-projection matrix is applied to each instance; the resulting saliency map is pooled using LSE-LBA.Source: §3.6 · p. 10

4 Experiments

The experiments evaluate weakly supervised classification and localization on NIH Chest X-rays, using patient-level splits and bounding-box annotations only for evaluation. Results show strong classification performance across abnormalities, while localization varies more with the pooling sharpness setting.

4.1 Dataset

  • The 112,120-image dataset uses official patient-level train and test splits, with manually annotated bounding boxes available only for a small test subset.It contains 30,805 patients and 14 labeled pathologies; the official training set is further split 75%/25% for training and validation.Source: §4.1 · pp. 10–11, Table 1 · p. 11

4.2 Training Procedures and Evaluation Metrics

  • Models use 512×512 inputs and Adam at 0.001, with validation average AUC selecting checkpoints; localization is evaluated without training boxes using continuous DICE instead of threshold-sensitive IoBB.IoBB changes substantially with the binarization threshold, whereas continuous DICE evaluates the probabilistic saliency map directly against the downsampled bounding box.Source: §4.2 · pp. 11–13, Table 2 · p. 12

4.3 Quantitative Results

  • The model beats the state of the art on 9 of 14 abnormalities and matches it on 3 others without extra data, but trails on fibrosis and hernia.Final AUCs are generally robust to r0, except for emphysema (optimal r0 = 0), fibrosis (r0 = 5), and hernia (r0 = 10).Source: Table 3 · p. 13, §4.3 · p. 13
  • Localization responds more to r0 than classification: stronger sharpness favors focal or patchy findings, while excessive sharpness degrades performance on diffuse findings.Small r0 favors diffuse cardiomegaly, infiltration, and pneumonia; stronger sharpness improves atelectasis and nodule localization, while too-large r0 degrades several diffuse abnormalities.Source: Table 3 · p. 13, §4.3 · p. 14

4.4 Qualitative Results

  • Figure 3 shows sharper saliency maps as r0 increases, while bounding boxes can overestimate infiltration and pneumonia regions and missed findings can appear to be false positives.The figure pairs images and ground-truth boxes with maps for r0 = 0, 5, and 10, and reports corresponding DICE scores.Source: Figure 3 · p. 14, §4.4 · p. 14

5 Conclusions

The conclusion presents a partial solution that combines multi-resolution and multi-instance learning with customized pooling. Evaluation on the largest publicly available chest X-ray dataset establishes state-of-the-art performance on most abnormalities and motivates exploring pre-training on another domain.

5 Conclusions

  • The proposed approach combines multi-resolution and multi-instance learning with customized pooling to produce diagnosis and higher-resolution pathology saliency maps.Source: §5 · p. 15
  • Evaluation on the largest publicly available chest X-ray dataset shows quantitative and qualitative efficacy, with state-of-the-art performance on a majority of the 14 abnormalities.Source: §5 · p. 15
  • The experiments suggest studying pre-training on data from another domain to improve performance.Source: §5 · p. 15

Related papers

Loading 1803.07703v1…