PaperlayerView exact revision on arXiv

Source-linked AI summary

Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification

Qingji Guan, Yaping Huang, Zhun Zhong, Zhedong Zheng, Liang Zheng, Yi Yang

arXiv:1801.09927v1cs.CV

TL;DR

Global-image CXR classification can be disrupted by irrelevant regions and image misalignment, motivating a method that combines local lesion cues with global context. AG-CNN uses attention-guided cropping and three complementary branches, and achieves state-of-the-art performance on ChestX-ray14.

  • Problem

    Global-image training may include irrelevant non-lesion noise and suffer from poor alignment when disease regions are small or images vary in acquisition conditions.

  • Method

    AG-CNN first trains a global branch, uses its attention heat map to crop a local lesion region for a local branch, and fuses both branches for fine-tuning.

  • Results

    Average AUC reaches 0.871 with DenseNet-121 on ChestX-ray14, exceeding the previous state of the art by 2.9%.

  • Takeaways & Limitations

    Combining global and local cues yields state-of-the-art accuracy on the ChestX-ray14 dataset.

Abstract

from arXiv · show

This paper considers the task of thorax disease classification on chest X-ray images. Existing methods generally use the global image as input for network learning. Such a strategy is limited in two aspects. 1) A thorax disease usually happens in (small) localized areas which are disease specific. Training CNNs using global image may be affected by the (excessive) irrelevant noisy areas. 2) Due to the poor alignment of some CXR images, the existence of irregular borders hinders the network performance. In this paper, we address the above problems by proposing a three-branch attention guided convolution neural network (AG-CNN). AG-CNN 1) learns from disease-specific regions to avoid noise and improve alignment, 2) also integrates a global branch to compensate the lost discriminative cues by local branch. Specifically, we first learn a global CNN branch using global images. Then, guided by the attention heat map generated from the global branch, we inference a mask to crop a discriminative region from the global image. The local region is used for training a local CNN branch. Lastly, we concatenate the last pooling layers of both the global and local branches for fine-tuning the fusion branch. The Comprehensive experiment is conducted on the ChestX-ray14 dataset. We first report a strong global baseline producing an average AUC of 0.841 with ResNet-50 as backbone. After combining the local cues with the global information, AG-CNN improves the average AUC to 0.868. While DenseNet-121 is used, the average AUC achieves 0.871, which is a new state of the art in the community.

I. INTRODUCTION

The paper addresses limitations of global-image CXR classification by introducing AG-CNN, which combines disease-specific local regions with global information. Its experiments report superior performance over state-of-the-art approaches on ChestX-ray14.

  • CXR classification is important because subtle lesion textures and complex pathologies can cause errors even for experienced radiologists.
  • Global-image learning may include substantial non-lesion noise because lesions can be small and unpredictably positioned, while CXR acquisition variations can produce misalignment.
  • AG-CNN uses global, local, and fusion branches to focus on disease-specific lesions while retaining global cues that may be lost locally.
  • The method follows a radiologist-like sequence: browse the whole image, inspect a cropped lesion region, then combine global and local information for classification.
  • AG-CNN is presented as a CNN training baseline and achieves superior performance over state-of-the-art approaches in comprehensive ChestX-ray14 experiments.

II. RELATED WORKS

Related work spans chest X-ray datasets, deep-learning methods for image analysis, and attention mechanisms for locating informative disease regions. AG-CNN uses attention-guided masks with image-level labels rather than requiring bounding-box annotations.

  • Chest X-ray datasets: Chest X-ray datasets vary in size, labeling, and available annotations, with ChestX-ray14 introduced as a larger labeled resource for thorax disease classification.
  • Deep learning for chest X-ray image analysis: Deep-learning studies have applied CNNs, ensembles, and related models to chest X-ray annotation, classification, and localization.
  • Attention models in medical image analysis: Attention-based medical-image methods seek informative regions because disease lesions often contain critical classification cues.
  • AG-CNN architecture: AG-CNN contains global and local classification branches plus a fusion branch that combines their pooled features.
  • Attention-guided localization: AG-CNN locates salient regions with an attention-guided mask and requires only image-level labels, unlike methods relying on bounding-box annotations.

III. THE PROPOSED APPROACH

AG-CNN uses separate global and local classification branches plus a fusion branch to combine whole-image and lesion-region information. The local region is generated from the global image, and the branches are trained sequentially before fusion fine-tuning.

  • Architecture: AG-CNN comprises global, local, and fusion branches for thorax disease classification.The global and local branches classify pathologies, while the fusion branch combines their representations for final prediction.
  • Global and local branches: The global branch processes the full CXR with a ResNet-50 backbone and predicts 15 pathology probabilities using a sigmoid output.Its parameters are optimized with binary cross-entropy loss.
  • Global and local branches: The local branch uses the attended crop as input and has the same convolutional structure as the global branch, but its weights are not shared.It is trained with the same normalization and optimization procedure as the global branch.
  • Training procedure: Training proceeds from global-branch learning to mask-based cropping, local-branch learning, and concatenation of global and local pooling outputs for fusion fine-tuning.The fusion branch connects the concatenated Pool5 outputs to a 15-dimensional fully connected layer for final classification.

B. Attention Guided Mask Inference

The attention-guided mask inference process derives a discriminative region from global-branch feature activations. Thresholding and connected-region extraction produce a crop that is resized for local-branch training.

  • Heat map and mask: The method builds a binary attention mask by thresholding a heat map derived from the global branch’s final convolutional feature maps.Positions whose heat-map values exceed threshold τ are assigned mask value 1.
  • Heat map and mask: The heat map uses the maximum absolute activation across channels at each spatial location to indicate activation importance.For ResNet-50, the final convolutional layer has K = 2,048 channels.
  • Heat map and mask: Different channel-wise statistics yield subtle numerical heat-map differences, but the paper reports no significant classification effect and uses the maximum-based formulation experimentally.The paper mentions L1 and L2 alternatives before selecting the formulation in Eq. 3.
  • Region extraction: The threshold τ controls attended-region size: larger values produce smaller regions, while smaller values produce larger regions.The selected mask points are enclosed by a maximum connected region whose coordinates define the crop.
  • Region extraction: The connected discriminative region is cropped from the global image and resized to the original image size for the local branch.With τ = 0.7, the inferred regions are reported as reasonably close to manually annotated lesion boxes, which are not used for training or testing.

C. Training Strategy of AG-CNN

AG-CNN uses a three-stage training scheme that separately trains global, local, and fusion branches. The local branch is guided by a mask inferred from the global branch, while the fusion branch combines both branches’ pooling outputs.

  • AG-CNN adopts a three-stage training scheme for its global, local, and fusion branches.
  • Stage I: Stage I fine-tunes an ImageNet-pretrained global branch using global images and normalizes its class probabilities.
  • Stage II: Stage II fine-tunes the local branch on a masked local image while keeping the global-branch weights fixed.
  • The local image is obtained through mask inference, while ground-truth lesion boxes are used only for demonstration.
  • Stage III: Stage III concatenates the Pool5 outputs of the global and local branches to fine-tune the fusion branch.
  • The highest-validation-AUC model from each stage is used for testing, and training-strategy variants may affect performance.

IV. EXPERIMENT

The experiment evaluates AG-CNN on ChestX-ray14 using an evaluation section that covers branch performance, fusion effectiveness, state-of-the-art comparison, and mask-threshold analysis.

  • The evaluation introduces the dataset, evaluation protocol, and experimental settings before analyzing branch performance and fusion.
  • The experiment compares AG-CNN with state-of-the-art methods and analyzes the impact of the mask-inference parameter.

A. Dataset and Evaluation Protocol

AG-CNN is evaluated on ChestX-ray14, a large frontal-view chest X-ray dataset with up to 14 pathology labels and a separate “No Finding” label. The protocol uses random 70%/10%/20% train-validation-test splits.

  • Dataset: ChestX-ray14 contains 112,120 frontal-view images from 30,805 unique patients.
  • Dataset: Of these images, 51,708 have up to 14 pathology labels, while the remainder are labeled “No Finding”.
  • Dataset: Lesion-area size varies substantially across pathologies in the dataset.
  • Evaluation protocol: The dataset is randomly divided into 70% training, 10% validation, and 20% testing subsets.
  • Evaluation protocol: Each image is represented by a 15-dimensional label vector, including 14 pathologies and a “No Finding” category.

B. Experimental Settings

The experimental settings include standard image augmentation and branch-specific optimization configurations. ROC curves compare global, local, and fusion branches across diseases and backbones.

  • Training resizes images to 256 × 256, applies random 224×224 crops and horizontal flips, and subtracts the ImageNet mean.
  • ROC curves compare the global, local, and fusion branches over all 14 pathologies with DenseNet-121 as backbone.
  • Fusing global and local information yields clear improvement in the corresponding ROC comparisons.
  • AG-CNN ROC curves are also presented for all 14 diseases using ResNet-50 and DenseNet-121 backbones.
  • Each branch is trained for 50 epochs with SGD, using backbone-specific mini-batch sizes and a learning rate reduced after 20 epochs.

C. Evaluation

Evaluation on ChestX-ray14 compares global, local, and fusion branches across backbones and training strategies. Fusion generally performs best, although localized cropping can lose information for some diseases.

  • Global branch: 0.841 average AUC is achieved by the ResNet-50 global baseline across 14 thorax diseases.
  • Local branch: 0.817 and 0.810 average AUCs are obtained by the local branch with ResNet-50 and DenseNet-121, respectively.The local branch is trained on cropped and resized lesion patches.
  • Limitations: Local lesion estimation may cause critical information loss, producing a 2.4% ResNet-50 gap between the global and local branches.The local branch may therefore suffer from inaccurate attention-area estimation.
  • Fusion branch: The fusion branch outperforms both global and local branches for ResNet-50 and DenseNet-121.With ResNet-50, its performance gaps over the global and local branches are 0.027 and 0.051, respectively; with DenseNet-121, it surpasses both branches for all 14 pathologies.
  • Class-level analysis: Nodule gains the largest improvement, 0.047, while AG-CNN is inferior to the global branch for Pneumonia.The Nodule result is associated with its small lesion areas; Pneumonia is an exception to the broader improvement pattern.
  • Training strategy: 0.868 average AUC is reached by AG-CNN with sequential three-branch training, improving the global baseline from 0.841.Sequential strategies perform better than the alternative training orders reported.
  • Heat-map analysis: Different heat-map statistics produce only subtle local-branch differences and do not significantly affect classification performance.

D. Parameter Analysis

Parameter analysis studies the threshold τ that defines local regions. AG-CNN performs best near τ = 0.7 and remains comparatively stable across a moderate threshold range.

  • Threshold role: τ defines the local regions and affects classification accuracy.
  • Validation analysis: 0.7 is the validation-set threshold at which AG-CNN achieves its best performance.The test-set results are therefore reported with τ = 0.7.
  • Threshold effects: At τ = 0.1, the local branch reaches 0.828 average AUC, close to the global branch result of 0.841.Small τ values preserve most attention-heat-map entries, making the cropped region close to the global image.
  • Fusion robustness: AG-CNN outperforms both branches by at least 1.7% at τ = 0.4 and 0.5, with its highest AUC above 0.866 for τ in [0.6, 0.8].

V. CONCLUSION

The conclusion presents AG-CNN as combining global and local cues through attention-guided lesion masking for thorax disease classification. Experiments support state-of-the-art ChestX-ray14 accuracy and relative insensitivity to parameter changes, while future work targets localization and annotation constraints.

  • Conclusion: AG-CNN combines global and local cues, using attention heat maps to mask important regions for local-branch training.
  • Conclusion: Combining global and local cues yields state-of-the-art accuracy on the ChestX-ray14 dataset.
  • Conclusion: The method is relatively insensitive to parameter changes.
  • Future work: Future work will investigate more accurate lesion localization and semi-supervised learning for sample-collection and annotation difficulties.
Loading 1801.09927v1…