PaperlayerView exact revision on arXiv

Source-linked AI summary

Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification

Qingji Guan, Yaping Huang, Zhun Zhong, Zhedong Zheng, Liang Zheng, Yi Yang

arXiv:1801.09927v1cs.CV

TL;DR

Global-image CXR classification can be disrupted by irrelevant regions and image misalignment, motivating a method that combines local lesion cues with global context. AG-CNN uses attention-guided cropping and three complementary branches, and achieves state-of-the-art performance on ChestX-ray14.

  • Problem

    Global-image training may include irrelevant non-lesion noise and suffer from poor alignment when disease regions are small or images vary in acquisition conditions.

    Source: §I · p. 1

  • Method

    AG-CNN first trains a global branch, uses its attention heat map to crop a local lesion region for a local branch, and fuses both branches for fine-tuning.

    Source: §I · p. 2, A. Structure of AG-CNN · p. 3, B. Attention Guided Mask Inference · p. 4

  • Results

    Average AUC reaches 0.871 with DenseNet-121 on ChestX-ray14, exceeding the previous state of the art by 2.9%.

    Source: C. Evaluation · p. 8

  • Takeaways & Limitations

    Combining global and local cues yields state-of-the-art accuracy on the ChestX-ray14 dataset.

    Source: §V · p. 9

Abstract

from arXiv · show

This paper considers the task of thorax disease classification on chest X-ray images. Existing methods generally use the global image as input for network learning. Such a strategy is limited in two aspects. 1) A thorax disease usually happens in (small) localized areas which are disease specific. Training CNNs using global image may be affected by the (excessive) irrelevant noisy areas. 2) Due to the poor alignment of some CXR images, the existence of irregular borders hinders the network performance. In this paper, we address the above problems by proposing a three-branch attention guided convolution neural network (AG-CNN). AG-CNN 1) learns from disease-specific regions to avoid noise and improve alignment, 2) also integrates a global branch to compensate the lost discriminative cues by local branch. Specifically, we first learn a global CNN branch using global images. Then, guided by the attention heat map generated from the global branch, we inference a mask to crop a discriminative region from the global image. The local region is used for training a local CNN branch. Lastly, we concatenate the last pooling layers of both the global and local branches for fine-tuning the fusion branch. The Comprehensive experiment is conducted on the ChestX-ray14 dataset. We first report a strong global baseline producing an average AUC of 0.841 with ResNet-50 as backbone. After combining the local cues with the global information, AG-CNN improves the average AUC to 0.868. While DenseNet-121 is used, the average AUC achieves 0.871, which is a new state of the art in the community.

I. INTRODUCTION

The paper addresses limitations of global-image CXR classification by introducing AG-CNN, which combines disease-specific local regions with global information. Its experiments report superior performance over state-of-the-art approaches on ChestX-ray14.

  • CXR classification is important because subtle lesion textures and complex pathologies can cause errors even for experienced radiologists.Source: §I · p. 1
  • Global-image learning may include substantial non-lesion noise because lesions can be small and unpredictably positioned, while CXR acquisition variations can produce misalignment.Source: §I · p. 1
  • AG-CNN uses global, local, and fusion branches to focus on disease-specific lesions while retaining global cues that may be lost locally.Source: §I · pp. 1–2
  • The method follows a radiologist-like sequence: browse the whole image, inspect a cropped lesion region, then combine global and local information for classification.Source: §I · p. 2
  • AG-CNN is presented as a CNN training baseline and achieves superior performance over state-of-the-art approaches in comprehensive ChestX-ray14 experiments.Source: §I · p. 2

II. RELATED WORKS

Related work spans chest X-ray datasets, deep-learning methods for image analysis, and attention mechanisms for locating informative disease regions. AG-CNN uses attention-guided masks with image-level labels rather than requiring bounding-box annotations.

Chest X-ray datasets

  • Chest X-ray datasets vary in size, labeling, and available annotations, with ChestX-ray14 introduced as a larger labeled resource for thorax disease classification.Source: §II · p. 2

Deep learning for chest X-ray image analysis

  • Deep-learning studies have applied CNNs, ensembles, and related models to chest X-ray annotation, classification, and localization.Source: §II · p. 2

Attention models in medical image analysis

  • Attention-based medical-image methods seek informative regions because disease lesions often contain critical classification cues.Source: §II · p. 2

AG-CNN architecture

  • AG-CNN contains global and local classification branches plus a fusion branch that combines their pooled features.Source: Figure 2 · p. 3

Attention-guided localization

  • AG-CNN locates salient regions with an attention-guided mask and requires only image-level labels, unlike methods relying on bounding-box annotations.Source: §II · p. 3

III. THE PROPOSED APPROACH

AG-CNN uses separate global and local classification branches plus a fusion branch to combine whole-image and lesion-region information. The local region is generated from the global image, and the branches are trained sequentially before fusion fine-tuning.

Architecture

  • AG-CNN comprises global, local, and fusion branches for thorax disease classification.The global and local branches classify pathologies, while the fusion branch combines their representations for final prediction.Source: A. Structure of AG-CNN · p. 3, §III · p. 3

Global and local branches

  • The global branch processes the full CXR with a ResNet-50 backbone and predicts 15 pathology probabilities using a sigmoid output.Its parameters are optimized with binary cross-entropy loss.Source: A. Structure of AG-CNN · p. 3
  • The local branch uses the attended crop as input and has the same convolutional structure as the global branch, but its weights are not shared.It is trained with the same normalization and optimization procedure as the global branch.Source: A. Structure of AG-CNN · p. 3

Training procedure

  • Training proceeds from global-branch learning to mask-based cropping, local-branch learning, and concatenation of global and local pooling outputs for fusion fine-tuning.The fusion branch connects the concatenated Pool5 outputs to a 15-dimensional fully connected layer for final classification.Source: A. Structure of AG-CNN · p. 4

B. Attention Guided Mask Inference

The attention-guided mask inference process derives a discriminative region from global-branch feature activations. Thresholding and connected-region extraction produce a crop that is resized for local-branch training.

Heat map and mask

  • The method builds a binary attention mask by thresholding a heat map derived from the global branch’s final convolutional feature maps.Positions whose heat-map values exceed threshold τ are assigned mask value 1.Source: B. Attention Guided Mask Inference · p. 4
  • The heat map uses the maximum absolute activation across channels at each spatial location to indicate activation importance.For ResNet-50, the final convolutional layer has K = 2,048 channels.Source: B. Attention Guided Mask Inference · p. 4
  • Different channel-wise statistics yield subtle numerical heat-map differences, but the paper reports no significant classification effect and uses the maximum-based formulation experimentally.The paper mentions L1 and L2 alternatives before selecting the formulation in Eq. 3.Source: B. Attention Guided Mask Inference · p. 4

Region extraction

  • The threshold τ controls attended-region size: larger values produce smaller regions, while smaller values produce larger regions.The selected mask points are enclosed by a maximum connected region whose coordinates define the crop.Source: B. Attention Guided Mask Inference · p. 4
  • The connected discriminative region is cropped from the global image and resized to the original image size for the local branch.With τ = 0.7, the inferred regions are reported as reasonably close to manually annotated lesion boxes, which are not used for training or testing.Source: B. Attention Guided Mask Inference · p. 4

C. Training Strategy of AG-CNN

AG-CNN uses a three-stage training scheme that separately trains global, local, and fusion branches. The local branch is guided by a mask inferred from the global branch, while the fusion branch combines both branches’ pooling outputs.

Stage I

Stage II

Stage III

IV. EXPERIMENT

The experiment evaluates AG-CNN on ChestX-ray14 using an evaluation section that covers branch performance, fusion effectiveness, state-of-the-art comparison, and mask-threshold analysis.

  • The evaluation introduces the dataset, evaluation protocol, and experimental settings before analyzing branch performance and fusion.Source: §IV · p. 5
  • The experiment compares AG-CNN with state-of-the-art methods and analyzes the impact of the mask-inference parameter.Source: §IV · p. 5

A. Dataset and Evaluation Protocol

AG-CNN is evaluated on ChestX-ray14, a large frontal-view chest X-ray dataset with up to 14 pathology labels and a separate “No Finding” label. The protocol uses random 70%/10%/20% train-validation-test splits.

Dataset

Evaluation protocol

B. Experimental Settings

The experimental settings include standard image augmentation and branch-specific optimization configurations. ROC curves compare global, local, and fusion branches across diseases and backbones.

  • Training resizes images to 256 × 256, applies random 224×224 crops and horizontal flips, and subtracts the ImageNet mean.Source: B. Experimental Settings · pp. 5–6
  • ROC curves compare the global, local, and fusion branches over all 14 pathologies with DenseNet-121 as backbone.Source: Figure 5 · p. 6
  • Fusing global and local information yields clear improvement in the corresponding ROC comparisons.Source: Figure 5 · p. 6
  • AG-CNN ROC curves are also presented for all 14 diseases using ResNet-50 and DenseNet-121 backbones.Source: Figure 6 · p. 6
  • Each branch is trained for 50 epochs with SGD, using backbone-specific mini-batch sizes and a learning rate reduced after 20 epochs.Source: B. Experimental Settings · p. 6

C. Evaluation

Evaluation on ChestX-ray14 compares global, local, and fusion branches across backbones and training strategies. Fusion generally performs best, although localized cropping can lose information for some diseases.

Global branch

  • 0.841 average AUC is achieved by the ResNet-50 global baseline across 14 thorax diseases.Source: C. Evaluation · p. 6

Local branch

  • 0.817 and 0.810 average AUCs are obtained by the local branch with ResNet-50 and DenseNet-121, respectively.The local branch is trained on cropped and resized lesion patches.Source: C. Evaluation · p. 6

Limitations

  • Local lesion estimation may cause critical information loss, producing a 2.4% ResNet-50 gap between the global and local branches.The local branch may therefore suffer from inaccurate attention-area estimation.Source: C. Evaluation · p. 7

Fusion branch

  • The fusion branch outperforms both global and local branches for ResNet-50 and DenseNet-121.With ResNet-50, its performance gaps over the global and local branches are 0.027 and 0.051, respectively; with DenseNet-121, it surpasses both branches for all 14 pathologies.Source: C. Evaluation · p. 7

Class-level analysis

  • Nodule gains the largest improvement, 0.047, while AG-CNN is inferior to the global branch for Pneumonia.The Nodule result is associated with its small lesion areas; Pneumonia is an exception to the broader improvement pattern.Source: C. Evaluation · p. 7

Training strategy

  • 0.868 average AUC is reached by AG-CNN with sequential three-branch training, improving the global baseline from 0.841.Sequential strategies perform better than the alternative training orders reported.Source: C. Evaluation · p. 8

Heat-map analysis

  • Different heat-map statistics produce only subtle local-branch differences and do not significantly affect classification performance.Source: C. Evaluation · p. 8

D. Parameter Analysis

Parameter analysis studies the threshold τ that defines local regions. AG-CNN performs best near τ = 0.7 and remains comparatively stable across a moderate threshold range.

Threshold role

Validation analysis

  • 0.7 is the validation-set threshold at which AG-CNN achieves its best performance.The test-set results are therefore reported with τ = 0.7.Source: D. Parameter Analysis · p. 9

Threshold effects

  • At τ = 0.1, the local branch reaches 0.828 average AUC, close to the global branch result of 0.841.Small τ values preserve most attention-heat-map entries, making the cropped region close to the global image.Source: D. Parameter Analysis · p. 9

Fusion robustness

  • AG-CNN outperforms both branches by at least 1.7% at τ = 0.4 and 0.5, with its highest AUC above 0.866 for τ in [0.6, 0.8].Source: D. Parameter Analysis · p. 9

V. CONCLUSION

The conclusion presents AG-CNN as combining global and local cues through attention-guided lesion masking for thorax disease classification. Experiments support state-of-the-art ChestX-ray14 accuracy and relative insensitivity to parameter changes, while future work targets localization and annotation constraints.

Conclusion

  • AG-CNN combines global and local cues, using attention heat maps to mask important regions for local-branch training.Source: §V · p. 9
  • Combining global and local cues yields state-of-the-art accuracy on the ChestX-ray14 dataset.Source: §V · p. 9
  • The method is relatively insensitive to parameter changes.Source: §V · p. 9

Future work

  • Future work will investigate more accurate lesion localization and semi-supervised learning for sample-collection and annotation difficulties.Source: §V · p. 9

Related papers

Loading 1801.09927v1…