PaperlayerView exact revision on arXiv

Source-linked AI summary

Convolutional Neural Networks for Histopathology Image Classification: Training vs. Using Pre-Trained Networks

Brady Kieffer, Morteza Babaie, Shivam Kalra, H. R. Tizhoosh

arXiv:1710.05726v1cs.CV

TL;DR

Limited labeled histopathology data raise the question of whether non-medical pre-trained CNNs can support classification as an alternative to training from scratch. The paper compares scratch training, pre-trained feature extraction, and fine-tuning on Kimia Path24. Pre-trained features are competitive with scratch training; fine-tuning yields no tangible VGG16 improvement but improves Inception-v3 results.

  • Problem

    Limited labeled histopathology data constrain training deep networks, prompting comparison of scratch training with non-medical pre-trained features and fine-tuning.

    Source: §I · p. 1

  • Method

    The study compares CNNs trained from scratch with non-medical pre-trained feature extractors, with and without fine-tuning, on Kimia Path24.

    Source: §I · p. 1, B. Pre-Trained CNN as a Feature… · p. 4

  • Results

    Pre-trained features are comparable to scratch training; VGG16 fine-tuning adds no improvement, while Inception-v3 fine-tuning improves results and reaches ηtotal = 56.98%.

    Source: §VI · p. 4, §V · p. 4

  • Takeaways & Limitations

    For Kimia Path24, pre-trained networks offer a competitive alternative to scratch training, while the reported value of fine-tuning differs by architecture.

    Source: §V · p. 4, §VI · p. 4

    The evaluation is on the Kimia Path24 dataset; the authors note that further work could use different architectures and more aggressive augmentation.

    Source: §VII · p. 6, §VI · p. 6

Abstract

from arXiv · show

We explore the problem of classification within a medical image data-set based on a feature vector extracted from the deepest layer of pre-trained Convolution Neural Networks. We have used feature vectors from several pre-trained structures, including networks with/without transfer learning to evaluate the performance of pre-trained deep features versus CNNs which have been trained by that specific dataset as well as the impact of transfer learning with a small number of samples. All experiments are done on Kimia Path24 dataset which consists of 27,055 histopathology training patches in 24 tissue texture classes along with 1,325 test patches for evaluation. The result shows that pre-trained networks are quite competitive against training from scratch. As well, fine-tuning does not seem to add any tangible improvement for VGG16 to justify additional training while we observed considerable improvement in retrieval and classification accuracy when we fine-tuned the Inception structure.

I. INTRODUCTION

Whole-slide histopathology images are large and difficult to analyze, while limited labeled data constrain training deep networks for pathology. This study examines training from scratch, non-medical pre-trained features, and fine-tuning for Kimia Path24, against a background of prior digital-pathology analysis and medical-image transfer learning.

I. INTRODUCTION

  • Limited labeled histopathology data constrain training deep networks, motivating comparisons of training from scratch, pre-trained feature extraction, and fine-tuning.Whole-scan images can reach 50,000×70,000 pixels and pose computer-vision challenges.Source: §I · p. 1
  • The study evaluates deep-network approaches on Kimia Path24 and reports that a pre-trained network may be the most viable option.The examined approaches are training from scratch, feature extraction, and fine-tuning.Source: §I · p. 1

II. BACKGROUND

  • Earlier digital-pathology work covered segmentation and retrieval, while medical-image studies had also reported success using networks pre-trained on non-medical images.Reported applications include CBIR for computer-aided diagnosis and ImageNet-pre-trained networks.Source: §II · pp. 1–2
  • This study focuses on evaluating non-medical pre-trained CNNs as feature extractors, with and without fine-tuning, for digital pathology.Source: §II · p. 2

III. DATA SET

Kimia Path24 contains 24 whole-slide images selected from more than 350 scans, with manually selected training and test patches for evaluation.

III. DATA SET

  • Kimia Path24 comprises 24 whole-slide images manually selected from more than 350 scans, depicting diverse body parts and texture patterns.Source: §III · p. 2
  • The dataset provides 27,055 training patches and 1,325 manually selected test patches, each 1000×1000 pixels.Test-patch locations were whitened to prevent their mistaken use for training.Source: §III · p. 2
  • All patches are grayscale, so the dataset neglects staining color.Patches represent 0.5mm×0.5mm areas.Source: §III · p. 2

A. Patch Selection

Patches are selected using a homogeneity threshold, then a small class-wise subset is used for fine-tuning and resized for CNN input.

A. Patch Selection

  • Kimia Path24 scans are divided into nonoverlapping 1000×1000-pixel patches; patches with homogeneity below 99% are ignored, a high threshold intended to ensure patches with significant texture are not ignored.Source: A. Patch Selection · p. 2
  • For fine-tuning, the authors do not use all patches to emulate settings without a large dataset; each patch is normalized to [0, 1] and downsized to 224×224 for CNN input.They note that more extensive training may destroy what a network has already learned.Source: A. Patch Selection · p. 2
  • Randomly selecting 100 patches per class yields a 2,400-patch training set, while the testing samples are relatively balanced and the training set is imbalanced.Different specimen sizes and frequencies are given as the main reasons for the imbalance.Source: A. Patch Selection · p. 2

B. Accuracy Calculation

The experiments use accuracy measures adopted from earlier work to enable comparison, including patch-to-scan and total accuracy.

B. Accuracy Calculation

  • The accuracy measures follow [17] for comparability, with 1,325 testing patches used in the experiments.Source: B. Accuracy Calculation · p. 2
  • Patch-to-scan accuracy, ηp, is defined from the set of retrieved images for an experiment, R.The supplied passage introduces the measure but does not provide its complete expression.Source: B. Accuracy Calculation · p. 2
  • Total accuracy is ηtotal = ηp × ηw; incorporating both accuracy measurements makes acceptable results more difficult to obtain.Source: B. Accuracy Calculation · p. 2

IV. METHODS

The experiments use VGG16 and Inception-v3 to compare pre-trained feature extraction with transfer learning for Kimia Path24. Fine-tuning updates selected network blocks and adds task-specific classification layers.

IV. METHODS

  • Pre-trained VGG16 and Inception-v3 are evaluated as feature extractors and after transferring some weights to the medical-imaging domain.The experiments use the architectures provided in Keras.Source: §IV · p. 2

A. Fine-Tuning Protocols

  • Fine-tuning retrains VGG16 block 5 and the final two Inception blocks, using a small learning rate and large momentum to limit weight changes.Training used data augmentation for 200 epochs with batch size 32.Source: A. Fine-Tuning Protocols · pp. 2, 4
  • Figure 1 shows selected training patches from each scan; each patch is 1000×1000 pixels, or 0.5 mm×0.5 mm.The displayed patches run from scan/class 0 to scan/class 23, top left to bottom right.Source: Figure 1 · p. 3, A. Fine-Tuning Protocols · p. 3
  • Figure 2 presents the Kimia Path24 training-set instance distribution on the left and the testing-set distribution on the right.The caption identifies which dataset split each side represents.Source: Figure 2 · p. 4

B. Pre-Trained CNN as a Feature Extractor

The feature-extractor setup uses a pre-trained network to produce deep features, then trains a linear SVM to classify them.

B. Pre-Trained CNN as a Feature Extractor

  • The last fully connected layer before classification is extracted as the feature vector from each pre-trained network.The networks were pre-trained on other image domains and are not used directly as classifiers.Source: B. Pre-Trained CNN as a Feature… · p. 4
  • A linear Support Vector Machine classifies the extracted deep features.The experiments used scikit-learn and LIBSVM for SVM training.Source: B. Pre-Trained CNN as a Feature… · p. 4
  • The pre-trained networks are used without fine-tuning in the feature-extraction setup.The passage names these variants FE-VGG16 and FE-Inception-v3.Source: B. Pre-Trained CNN as a Feature… · p. 4

C. Fine-Tuned CNN as a Classifier

For transfer learning, the authors replace the pre-trained networks’ fully connected layers with task-specific layers initialized using bottleneck features, then fine-tune selected convolutional blocks.

C. Fine-Tuned CNN as a Classifier

  • Bottleneck features initialize a task-specific fully connected classifier before it is attached to the convolutional layers for fine-tuning.The VGG16 classifier uses a 256-unit dense ReLU layer and softmax output; Inception-v3 uses 1024 dense ReLU units and softmax.Source: C. Fine-Tuned CNN as a Classifier · p. 4
  • The transfer-learned VGG16 and Inception-v3 networks are used to classify the test patches after their selected convolutional blocks are trained.For Inception-v3, training updates the final two inception blocks.Source: C. Fine-Tuned CNN as a Classifier · p. 4
  • Class activation mappings for the fine-tuned Inception-v3 network are shown for randomly selected test patches in Figure 3.The passage identifies the network and samples represented in the figure.Source: C. Fine-Tuned CNN as a Classifier · p. 4

V. RESULTS

The results show comparable performance for VGG16 and CNN1 across training from scratch, feature extraction, and fine-tuning. Inception-v3 performs better after transfer learning than as a feature extractor.

V. RESULTS

  • ηtotal = 56.98% for TL-Inception-v3, the best result; transfer-learned Inception-v3 outperforms its feature-extractor variant.The authors report similar Inception-v3 results otherwise.Source: §V · p. 4
  • VGG16 and CNN1 results are quite similar across training from scratch, pre-trained feature extraction, and fine-tuning.The comparison covers all three training approaches named in the results passage.Source: §V · p. 4
  • The authors suggest Inception-v3 with a linear SVM as an option because it gives similar results without extra effort.They also note that minimally updating pre-trained weights is not time-consuming.Source: §V · p. 4

VI. DISCUSSIONS

Pre-trained features performed competitively with scratch training on Kimia Path24, while fine-tuning helped Inception-v3 but not VGG16. The discussion also treats possible scratch-training superiority with abundant data and compute as speculative, and identifies architecture and augmentation as possible improvement paths.

VI. DISCUSSIONS

  • ≈8.74% overall performance increase was reported for the pre-trained-feature approach on Kimia Path24, which also performed comparably to scratch-trained networks.Table 1 compares scratch CNN1, pre-trained feature extractors, and fine-tuned networks; its caption notes that best scores are highlighted.Source: §VI · p. 4, Table 1 · p. 5
  • Fine-tuning VGG16 added no improvement over feature extraction, whereas fine-tuning Inception-v3 produced an immediate improvement.The discussion reports the Inception-v3 improvement alongside the VGG16 result.Source: §VI · p. 4
  • Scratch CNNs might deliver better results with millions of histopathology images and efficient training hardware, but the authors call this speculative for medical imaging.The discussion frames this as a possibility, not an experimental finding.Source: §VI · pp. 4–5
  • Activation maps show responses to distinct within-class patch patterns, while different architectures and more aggressive augmentation are suggested as possible improvements.The discussion says the maps indicate the model had begun learning higher-level structures within individual patches.Source: §VI · pp. 5–6

VII. CONCLUSIONS

Histopathology retrieval and classification are useful but challenging. On Kimia Path24, feature extractors and transfer-learned networks improve classification accuracy over a CNN trained from scratch and compete with state-of-the-art methods reported in the literature [17].

  • Both feature-extractor and transfer-learned models improve Kimia Path24 classification accuracy over a CNN trained from scratch and compete with reported state-of-the-art methods [17].Source: §VII · p. 6
  • The comparatively low scratch-trained CNN performance may reflect unsuitable architecture, too few training images, and/or the task’s difficulty with high-resolution, variable images.Source: §VII · p. 6
  • Further work could compare other architectures, use more aggressive data augmentation, or increase the Kimia Path24 training sample size.Source: §VII · p. 6
Loading 1710.05726v1…