PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning

Hoo-Chang Shin, Holger R. Roth, Mingchen Gao, Le Lu, Ziyue Xu, Isabella Nogues, Jianhua Yao, Daniel Mollura, Ronald M. Summers

arXiv:1602.03409v1cs.CV

TL;DR

Medical CADe lacks ImageNet-scale annotated datasets, leaving CNN architecture, dataset scale, spatial context, and transfer learning insufficiently studied. The paper evaluates these factors across lymph-node detection and ILD classification, surpassing prior mediastinal LN detection results with 86% sensitivity at 3 false positives per patient and reporting first patient-level five-fold ILD results.

  • Problem

    Medical imaging lacks large-scale, comprehensively annotated datasets comparable to ImageNet, while ImageNet fine-tuning for medical image datasets remains underexplored.

  • Method

    The paper evaluates CNN architectures, dataset scale, spatial context, and ImageNet transfer learning across lymph-node detection and ILD classification.

  • Results

    86% sensitivity at 3 false positives per patient surpassed prior mediastinal LN detection results, while ILD classification was evaluated under patient-level five-fold cross-validation for the first time.

  • Takeaways & Limitations

    CNN architecture and dataset resampling matter for LN detection, while training from scratch or ImageNet fine-tuning consistently outperforms using off-the-shelf CNN features.

  • Takeaways & Limitations

    The ILD dataset is partially labeled, with few normal or healthy CT slices and unlabeled disease regions that cannot be simply classified as normal.

Abstract

from arXiv · show

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and the revival of deep CNN. CNNs enable learning data-driven, highly representative, layered hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, with 85% sensitivity at 3 false positive per patient, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

I. INTRODUCTION

The introduction frames large annotated datasets and deep CNNs as drivers of image-recognition progress, while medical imaging lacks similarly comprehensive datasets. This work evaluates CNN architecture, dataset scale, spatial context, and ImageNet transfer learning for lymph-node detection and ILD classification, reporting improved mediastinal LN detection and comparative findings on training strategies.

  • Motivation: ImageNet provides more than 1.2 million categorized natural images across 1000+ classes, supporting CNN backbones that improve object detection and image segmentation.The introduction emphasizes large-scale, representative, well-annotated datasets as crucial for accurate and generalizable data-driven models.
  • Existing approaches: Medical image classification commonly uses CNNs trained from scratch, off-the-shelf CNN features, or unsupervised pre-training followed by supervised fine-tuning.These approaches are presented as three major techniques for employing CNNs in medical image classification.
  • Study scope: The study evaluates CNN width from 5 thousand to 160 million parameters, varying depth, dataset scale, spatial image context, and ImageNet transfer learning for CADe.The paper focuses on when and why fine-tuning pre-trained ImageNet CNNs can help medical imaging tasks.
  • Results: 86% sensitivity at 3 false positives per patient was achieved for mediastinal LN detection, exceeding prior state-of-the-art sensitivities of 78% and 70%.The 78% result came from stacked shallow learning, while the 70% result came from a CNN.
  • Results: CNNs trained from scratch or fine-tuned from ImageNet consistently outperform CNNs using only off-the-shelf features in both LN and ILD classification.The experiments also identify CNN architecture and dataset resampling as critical for LN detection when labeled data are sufficient and spatial contexts are local.

II. DATASETS AND RELATED WORK

The study evaluates CNNs on thoraco-abdominal lymph-node detection and interstitial-lung-disease classification using 2.5D CT views or 2D axial slices. Its datasets and preprocessing capture multiscale spatial context, encode CT windows as RGB channels, and confront partial labeling in the ILD data.

  • Datasets and tasks: The study uses randomly sampled 2.5D CT views for lymph-node detection and 2D CT slices for interstitial-lung-disease detection.Lymph-node performance is evaluated separately in mediastinal and abdominal regions.
  • Thoracoabdominal Lymph Node Datasets: The lymph-node dataset contains 388 mediastinal nodes from 90 CT patients and 595 abdominal nodes from 86 patients, with candidates sampled around annotated or detected centers.The adopted preparation protocol uses 30mm-to-45mm fields of view for comparison with prior work.
  • Thoracoabdominal Lymph Node Datasets: 35mm-to-128mm fields of view, soft-tissue window [-100, 200HU], and 64 × 64 pixel resampling provide multiple spatial scales of lymph-node image context.Axial, coronal, and sagittal views are randomly resampled and encoded as RGB images.
  • Interstitial Lung Disease Dataset: The ILD dataset contains 905 image slices from 120 patients annotated for six tissue types: healthy, emphysema, ground glass, fibrosis, micronodules, and consolidation.Slice-level classification predicts presence or absence of each ILD class from an axial CT slice.
  • Interstitial Lung Disease Dataset: Few ILD slices are labeled healthy, while many remaining slices cannot be treated as normal because disease regions may be unlabeled.This partial labeling is identified as a main limitation of the publicly available ILD database.
  • Interstitial Lung Disease Dataset: Encoding lung, high-attenuation, and low-attenuation CT windows into RGB channels improves classification over a single window, while lung segmentation shows no significant performance difference between tested setups.The CNN learns ILD-region localization through selectively weighted receptive fields during classification.

III. METHODS

The study examines three factors in building effective deep CNN models for computer-aided detection: CNN architectures, dataset characteristics, and transfer learning from non-medical to medical image domains.

  • III. METHODS: The methods evaluate CNN architectures, dataset characteristics, and transfer learning as key elements for effective deep CNN models in CADe.Dataset characteristics concern whether additional training data or better models are needed for object detection.

A. Convolutional Neural Network Architectures

The section compares CifarNet, AlexNet, and GoogLeNet for CADe, adapting ImageNet architectures to medical detection and classification problems. It emphasizes substantial differences in model complexity, input dimensions, and architectural depth.

  • Architecture comparison: Medical-imaging CNN models are at least 2 ∼5 orders of magnitude smaller than AlexNet, with some complex models containing about 150K or 15K parameters.The study therefore evaluates architectures spanning substantially different model sizes and numbers of layers.
  • CifarNet: CifarNet serves as the LN-detection baseline, using 32 × 32 × 3 image patches and about 0.15 million free parameters.Its architecture has three convolution layers, three pooling layers, and one fully connected layer.
  • Architecture comparison: AlexNet and GoogLeNet are modified from ImageNet classification for the CADe problems and datasets, originally taking 256 × 256 × 3-pixel images.CifarNet instead always accepts 32 × 32 × 3 patches.
  • GoogLeNet: GoogLeNet is significantly more complex and deeper than previous architectures, introducing Inception modules that concatenate filters of different sizes and dimensions.Its architecture has two convolution layers, two pooling layers, and nine Inception layers; each Inception layer contains six convolution layers and one pooling layer.

B. ImageNet: Large Scale Annotated Natural Image Dataset · C. Training Protocols and Transfer Learning

ImageNet provides a large, densely annotated natural-image pretraining resource, while the paper evaluates scratch training, fine-tuning, and fixed-feature transfer protocols for medical CADe tasks. Transfer learning is motivated by annotation scarcity and reported to yield the best performance among the tested strategies.

  • B. ImageNet: Large Scale Annotated Natural Image Dataset: ImageNet contains more than 1.2 million 256 × 256 images across 1000 object categories, with more than 1000 training images per class.Its labels are largely crowd-sourced and human-inspected, and the database follows the WordNet hierarchy.
  • B. ImageNet: Large Scale Annotated Natural Image Dataset: ImageNet is described as the largest image dataset among standard visual-recognition datasets.Its categories include objects such as sea snakes, sandwiches, vases, and leopards.
  • C. Training Protocols and Transfer Learning: Scratch-trained CNN parameters use random Gaussian initialization, 30 epochs, mini-batches of 50, momentum 0.9, weight decay 0.0005, and scheduled learning-rate reductions.Training uses Caffe on NVIDIA K40 GPUs, with the learning rate reduced tenfold every 10 epochs from 0.01.
  • C. Training Protocols and Transfer Learning: The transfer-learning strategy yields the best performance results among the reported training strategies.AlexNet and GoogLeNet can be trained either from scratch or fine-tuned from pretrained models.
  • C. Training Protocols and Transfer Learning: Transfer learning is hypothesized to improve medical image recognition because ImageNet is large and well annotated, whereas collecting and labeling many medical images remains difficult.The hypothesis acknowledges the disparity between natural and medical images.
  • C. Training Protocols and Transfer Learning: Fine-tuning updates all CNN layers except the last at a learning rate 10 times smaller than default, while the final layer is randomly initialized and trained for new CADe categories.The last layer keeps the original learning rate of 0.01; the resulting models are labeled AlexNet-RI, AlexNet-TL, GoogLeNet-RI, and GoogLeNet-TL.
  • C. Training Protocols and Transfer Learning: The off-the-shelf AlexNet-ImNet model fixes convolutional and fully connected parameters as deep image extractors and trains only the final classifier.The paper also notes reported effectiveness of CNN representation transfer across RGB-to-depth, natural-image-to-CT/MRI, neuroimaging, and ultrasound settings.

IV. EVALUATIONS AND DISCUSSIONS · A. Thoracoabdominal Lymph Node Detection

The evaluation compares nine CNN configurations for thoracoabdominal lymph-node detection using three-fold patient-disjoint cross-validation and random-view probability aggregation. Transfer learning, larger spatial context, and ensemble averaging improve performance, with GoogLeNet-RI-L achieving strong mediastinal results.

  • IV. EVALUATIONS AND DISCUSSIONS: Nine CNN configurations are evaluated on mediastinal and abdominal lymph-node detection using publicly available datasets.The configurations include CifarNet, AlexNet variants, GoogLeNet variants, and a combined model.
  • A. Thoracoabdominal Lymph Node Detection: Three-fold cross-validation splits patients into disjoint folds, while averaged probabilities from multiple random 2.5D views produce each lymph-node candidate’s score.The evaluation follows random-view aggregation by averaging CNN-generated probabilities.
  • A. Thoracoabdominal Lymph Node Detection: 64 × 64-pixel patches are up-sampled to 256 × 256 pixels for several CNNs, while reduced filters, kernels, padding, and strides adapt modified models to smaller inputs.These architectural reductions accommodate medical-image patch sizes while helping prevent over-fitting.
  • A. Thoracoabdominal Lymph Node Detection: Detection performance is reported with FROC-AUC and TPR/3FP, using FROC curves averaged across three-fold cross-validation.TPR/3FP denotes sensitivity at three false positives per patient.
  • A. Thoracoabdominal Lymph Node Detection: CifarNet, AlexNet-ImNet, and GoogLeNet-RI-H are generally least competitive, reflecting under-training or over-fitting on the more variable lymph-node datasets.The datasets use larger 35mm-128mm fields of view than the 30mm-45mm range in the cited prior study.
  • A. Thoracoabdominal Lymph Node Detection: 0.81 versus 0.61 TPR/3FP in mediastinum and 0.70 versus 0.48 TPR/3FP in abdomen show GoogLeNet-TL-H’s improvement over GoogLeNet-RI-H.The comparison indicates that transfer learning substantially improves the deeper GoogLeNet model under limited training data.
  • A. Thoracoabdominal Lymph Node Detection: A five-CNN probability-averaging ensemble matches or slightly exceeds the best individual model on mediastinal or abdominal detection, respectively.Aggressive random-view sampling across larger fields of view captures more spatial context and contributes to the best reported results.
  • A. Thoracoabdominal Lymph Node Detection: GoogLeNet-RI-L obtains AUC=0.95 and 0.85 TPR/3FP for mediastinal detection, exceeding prior AUC=0.92 and 0.70 TPR/3FP results.The CNN approach requires lymph-node locations rather than annotated segmentation masks used by the cited stacked shallow-learning method.

B. Interstitial Lung Disease Classification

The study evaluates ILD classification with patient-level five-fold cross-validation across patch- and slice-level tasks, finding patch classification easier and ImageNet transfer learning beneficial for the challenging slice-level setting. It classifies six ILD categories and reports comparable medium-to-high patch-level F-scores across five- and ten-fold protocols.

  • Evaluation protocol: Five-fold cross-validation uses patient-level splits for ILD classification, providing a more clinically informative evaluation than leave-one-out validation.The study evaluates both patch- and slice-level classification tasks.
  • Patch- and slice-level classification: Patch-level ILD classification is less challenging than slice-level classification because manually annotated regions support substantially more sampled training patches.Up to 100 image patches can be sampled per manually annotated region of interest.
  • ILD classes: Six ILD classes comprise normal, emphysema, ground glass, fibrosis, micronodules, and consolidation.The consolidation class is added to the five classes used in prior studies.
  • Patch-level classification: Patch-CV10 and Patch-CV5 produce similar medium-to-high F-scores, suggesting consistent patch-level performance across the two cross-validation protocols.F1-scores are reported for 32×32 pixel patches extracted from manually annotated regions using GoogLeNet-TL under five-fold cross-validation.
  • Slice-level classification: 0.57 accuracy is obtained by GoogLeNet-TL for slice-level classification despite data augmentation, while ImageNet transfer learning consistently improves performance, including AlexNet-TL at 0.46.Slice-level learning is restricted to 905 CT slices with tagged ILD labels, making the task especially challenging.

C. Evaluation of Five CNN Models using ILD Classification

The evaluation compares five CNN architectures for ILD slice- and patch-level classification, finding higher accuracy with increasing model complexity. Transfer learning is applied only to slice classification, while LOO validation substantially outperforms five-fold cross-validation.

  • Model evaluation: Five CNN architectures—CifarNet, AlexNet, Overfeat, VGGNet, and GoogLeNet—are evaluated for ILD slice- and patch-level classification.The models are ordered from lower to higher complexity in the reported comparison.
  • Model evaluation: Classification accuracy increases as CNN complexity rises from CifarNet through GoogLeNet for both ILD slice- and patch-level tasks.The reported ascending order is CifarNet, AlexNet, Overfeat, VGGNet, and GoogLeNet.
  • Transfer learning: Transfer learning is used only for ILD slice-level classification.The evaluation bases this choice on the analysis in Section IV-B.
  • Validation protocol: 0.46 to 0.867 accuracy is reported for AlexNet-TL, while GoogLeNet-TL rises from 0.57 to 0.902 under LOO rather than five-fold cross-validation.The passage describes LOO quantitative performance as remarkably better than CV-5, including these ILD slice-level comparisons.

D. Training with “Equal Prior” vs. “Biased Prior”

The study examines training under equal versus biased class priors for ILD and lymph-node detection, where medical datasets contain substantial class imbalances. GoogLeNet results are therefore compared on balanced validation sets to avoid unfair accuracy assessments caused by biased validation distributions.

  • Dataset imbalance: Medical datasets are often biased, with healthy samples or images per class outnumbering diseased or other classes.Fibrosis samples are about 3.5 times more numerous than emphysema samples, while non-LNs outnumber LNs by 3–4 times.
  • Dataset imbalance: Different sampling or resampling rates are routinely applied to ILD and LN detection to balance the data.
  • Evaluation protocol: GoogLeNet classification results after equal- or biased-prior training are compared on balanced validation sets.Evaluating on a biased validation set would cause an unfair assessment of classifier performance.
  • Evaluation protocol: 70% accuracy can result from predicting every image patch as “non-LN” on a biased set with 3.5 times as many non-LN samples as LN samples.

V. ANALYSIS VIA CNN LEARNING TRACES & LULVISUALIZATION

The section analyzes CNN learning traces and visualizations to explain why transfer learning improves computer-aided detection. It shows that fine-tuning affects learned filters and produces more consistent optimization behavior.

  • Transfer learning is analyzed through CNN visualization to explain its benefits for computer-aided detection performance.The analysis examines why transfer learning is beneficial for CAD applications.
  • Thoracoabdominal LN Detection: AlexNet-TL (256x256) learns higher-order contrast- or edge-preserving patterns, whereas four non-transfer-learning architectures learn somewhat blurry first-layer filters.The compared architectures are AlexNet-RI (256x256), AlexNet-RI (64x64), GoogLeNet-RI (256x256), and GoogLeNet-RI (64x64).
  • ILD classification: In ILD classification, AlexNet-RI’s training loss decreases while validation loss increases and validation accuracy shows little improvement before plateauing.The traces compare training loss, validation loss, and validation accuracy for AlexNet-RI and AlexNet-TL.
  • ILD classification: The last pooling-layer activation map summarizes the full input image by highlighting activated relative locations or neural reception fields.AlexNet has 256 (6x6) reception fields, and the comparison uses ImageNet pre-trained AlexNet and AlexNet-TL.

VI. FINDINGS AND FUTURE DIRECTIONS

The findings show that deep CNNs, transfer learning, and dataset scale are central to CADe performance, while limited annotation remains a major bottleneck. Future progress should balance model development with progressively larger datasets and improved feature integration or fine-tuning.

  • Findings: Deep CNN architectures with 8, even 22 layers can be useful for CADe problems with limited training datasets, despite medical models often being 2 ∼5 orders of magnitude smaller.The finding supports using substantially deeper architectures even when available medical imaging datasets are limited.
  • Findings: The trade-off between better learning models and more training data should be carefully considered when optimizing CADe solutions, including mediastinal and abdominal LN detection.Model quality and dataset scale are presented as competing considerations in searching for an optimal solution.
  • Future Directions: Limited datasets can bottleneck CADe advancement, making progressively larger, well-annotated datasets at least as crucial as developing new algorithms.The paper points to the steady growth of computer-vision datasets as an example of dataset-driven progress.
  • Findings: Transfer learning from large-scale annotated natural-image datasets such as ImageNet was consistently beneficial across the experiments.This result motivates cross-dataset CNN learning in medical imaging, including combining the ILD and LTRC datasets.
  • Future Directions: Off-the-shelf deep CNN features may be improved by combining complementary hand-crafted features, training CNNs from scratch, or better fine-tuning them on target medical datasets.These alternatives were evaluated in the paper as ways to improve applications of deep CNN image features to CADe.

VII. CONCLUSION

The paper extensively evaluates CNN architecture, dataset characteristics, and transfer learning across thoracoabdominal lymph node detection and interstitial lung disease classification. It also presents empirical analyses, model visualizations, and conclusive insights intended to generalize to other applications.

  • Conclusion: The study evaluates three previously understudied factors: CNN architecture, dataset characteristics, and transfer learning.These factors are examined through extensive empirical evaluation.
  • Conclusion: The evaluation covers two computer-aided diagnosis applications: thoracoabdominal lymph node detection and interstitial lung disease classification.The paper assesses CNN performance on both applications.
  • Conclusion: The paper combines empirical evaluation, CNN model visualization, and CNN performance analysis to derive conclusive insights.The passage states that these insights can be generalized to th…
Loading 1602.03409v1…