PaperlayerView exact revision on arXiv

Source-linked AI summary

Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides

Abtin Riasatian, Morteza Babaie, Danial Maleki, Shivam Kalra, Mojtaba Valipour, Sobhan Hemati, Manit Zaveri, Amir Safarpoor, Sobhan Shafiei, Mehdi Afshari, Maral Rasoolijaberi, Milad Sikaroudi, Mohd Adnan, Sultaan Shah, Charles Choi, Savvas Damaskinos, Clinton JV Campbell, Phedias Diamandis, Liron Pantanowitz, Hany Kashani, Ali Ghodsi, H. R. Tizhoosh

arXiv:2101.07903v1eess.IV

TL;DR

The paper tackles the limited availability of labeled pathology images for training domain-specific representations. It develops KimiaNet from DenseNet using high-cellularity mosaics and weakly labeled TCGA patches, then evaluates it across public search and classification datasets. KimiaNet outperforms the original DenseNet and smaller CBR networks as a histopathology feature extractor.

Abstract

from arXiv · show

Feature vectors provided by pre-trained deep artificial neural networks have become a dominant source for image representation in recent literature. Their contribution to the performance of image analysis can be improved through finetuning. As an ultimate solution, one might even train a deep network from scratch with the domain-relevant images, a highly desirable option which is generally impeded in pathology by lack of labeled images and the computational expense. In this study, we propose a new network, namely KimiaNet, that employs the topology of the DenseNet with four dense blocks, fine-tuned and trained with histopathology images in different configurations. We used more than 240,000 image patches with 1000x1000 pixels acquired at 20x magnification through our proposed "highcellularity mosaic" approach to enable the usage of weak labels of 7,126 whole slide images of formalin-fixed paraffin-embedded human pathology samples publicly available through the The Cancer Genome Atlas (TCGA) repository. We tested KimiaNet using three public datasets, namely TCGA, endometrial cancer images, and colorectal cancer images by evaluating the performance of search and classification when corresponding features of different networks are used for image representation. As well, we designed and trained multiple convolutional batch-normalized ReLU (CBR) networks. The results show that KimiaNet provides superior results compared to the original DenseNet and smaller CBR networks when used as feature extractor to represent histopathology images.

I. INTRODUCTION

The paper addresses the challenge of representing complex, large histopathology WSIs by investigating domain-specific fine-tuning and training of deep networks. It proposes KimiaNet and evaluates its features across pathology search and classification tasks.

  • Histopathology WSIs require compact, expressive representations because they are large and contain complex patterns.Source: §I · p. 1
  • KimiaNet features are evaluated for search and classification on three public datasets and compared with original DenseNet and smaller trained networks.Source: §I · p. 2
  • Prior pathology studies used pre-trained, fine-tuned, or trained networks for classification, retrieval, and WSI-level diagnosis, including approaches based on VGG-19, Inception V3, ResNet, and recurrent networks.Source: §II · p. 2
  • Deep embeddings from pre-trained networks are widely used in pathology, but the effects of fine-tuning and training from scratch across tasks remain underexplored.Source: §II · p. 3
  • The study proposes cellMosaic, a high-cellularity patch-selection method that enables weak WSI-level labels from TCGA to train a densely connected network.Source: §II · p. 3

III. KIMIANET - DATA AND TRAINING

KimiaNet uses DenseNet as its architectural basis because DenseNet is established, compact, and suitable for image representation. The paper positions architecture customization as necessary for application-specific pathology tasks.

  • KimiaNet is based on DenseNet, a densely connected architecture composed of dense blocks preceded by convolutional and pooling layers.Source: §III · p. 3
  • The paper connects architecture customization with application-oriented categorization and end-user awareness in sensitive tasks.Source: §III · p. 3
  • DenseNet was selected because it is a dependable topology with a compact footprint for image representation.Source: §III · p. 3

A. Public Image Datasets

The study uses three public pathology datasets spanning whole-slide and labeled-patch data. TCGA supplies the large multi-organ WSI resource used to prepare training data.

  • The experiments use public TCGA, endometrial cancer, and colorectal cancer datasets.TCGA contains WSIs, while the endometrial and colorectal datasets contain labeled patches.Source: A. Public Image Datasets · p. 3
  • TCGA provides 30,072 WSIs across 25 anatomic sites and 32 cancer subtypes, with 29,120 files readable at 20× magnification.Source: 1) TCGA Images · p. 3
  • Figure 1 presents the DenseNet architecture used for KimiaNet.Source: Figure 1 · p. 4
  • The endometrial dataset contains 3,302 images across four tissue classes, with patches extracted at 20× or 10× magnification.Source: 2) Endometrium dataset · p. 4

3) Colorectal Cancer Dataset:

The paper processes colorectal and TCGA imagery as patches and uses modified Yottixel mosaics to derive high-cellularity training representations from weakly labeled WSIs.

C. Training Data

The training dataset was constructed from TCGA slides using detailed diagnostic groupings, selected magnification and patch settings, and cellularity-based patch filtering. KimiaNet was trained in four configurations with progressively more DenseNet blocks updated.

Dataset construction

  • Two TCGA primary-diagnosis classes were removed because morphology information or sufficiently represented detailed groups were unavailable.The remaining data were organized using detailed-label combinations and established tumour-type categories.Source: C. Training Data · p. 5

Patch settings

  • 20× magnification was chosen for detailed histology patterns, while patch size and magnification were set using empirical evidence and computational convenience.Pathologists use 10× for gross features, 20× for detailed histology, and 40× for fine nuclear and cellular details.Source: C. Training Data · p. 5

Patch selection

  • Cellularity was used to remove many benign or healthy patches because WSI-level diagnosis labels did not directly label individual training patches.The approach targeted patches with unusually high numbers of cell nuclei, which are associated with uncontrolled cell growth in carcinomas.Source: C. Training Data · p. 6

Model configurations

  • KimiaNet-I through KimiaNet-IV trained the last one, two, three, or all DenseNet-121 blocks, respectively.Each configuration used a different extent of fine-tuning or retraining within the same DenseNet-121 architecture.Source: D. Training · p. 6

Training outcomes

  • KimiaNet-IV achieved the highest accuracy, and accuracy generally increased as more DenseNet blocks were fine-tuned.Convergence was observable after about 10 epochs in most experiments.Source: D. Training · p. 6

IV. EXPERIMENTS

The experiments assessed KimiaNet features for generalization and compared them with DenseNet features using search-based evaluation. The broader setup also considered segmentation, patching, clustering, and network hyperparameters.

Evaluation setup

  • KimiaNet and original DenseNet features were extracted from the last pooling layer and evaluated across three datasets for generalization.Search used k-NN with k = 3 to retrieve the most similar feature barcodes.Source: §IV · p. 6

Experimental considerations

  • Deep-network solutions for histopathology gigapixel images require tuning network parameters and additional segmentation, patching, and clustering parameters.Source: §IV · p. 7

A. TCGA Experiments: Classification through Search

TCGA experiments evaluated feature distinctiveness through horizontal tumour-type search and vertical primary-diagnosis search. KimiaNet outperformed DenseNet across the reported search measures.

Search tasks

Evaluation metrics

Horizontal search

Vertical search

B. Endometrium Data Experiments: Classification

The endometrium experiments compare KimiaNet features with other deep feature extractors for image classification. KimiaNet is evaluated alongside DenseNet and fine-tuned VGG-19, with confusion matrices reported for DenseNet and KimiaNet.

B. Endometrium Data Experiments: Classification

E. CBR Nets: Small versus large

The paper benchmarks smaller CBR networks against KimiaNet and other feature representations across search and classification visualizations. KimiaNet achieves stronger and more consistent TCGA search results than the compared smaller topologies.

E. CBR Nets: Small versus large

  • Small CBR networks can produce results comparable to larger ImageNet models in some medical-image applications.Prior work compared a roughly 2M-weight Small CBR network with much larger ResNet50 and Inception-V3 models.Source: E. CBR Nets: Small versus large · pp. 8–9
  • CBR feature performance was very low without the additional dense layer.Source: E. CBR Nets: Small versus large · p. 9
  • KimiaNet achieved a 30% accuracy difference over the second-best topology in horizontal TCGA search.The second-best topology, LargeT, had around 1.5M more parameters than KimiaNet.Source: E. CBR Nets: Small versus large · p. 10
  • KimiaNet achieved the maximum accuracy for all 12 horizontal-search classes, while no other network did so for any class.Source: E. CBR Nets: Small versus large · p. 10
  • KimiaNet’s average vertical-search F1 score was 10% higher than the second-best LargeW network.KimiaNet also showed a low standard deviation of F1 scores relative to the compared networks.Source: E. CBR Nets: Small versus large · p. 10

V. SUMMARY AND CONCLUSIONS

The paper addresses pathology image representation by adapting DenseNet to weakly labeled, high-resolution TCGA patches through KimiaNet and a high-cellularity mosaic. It reports improved performance over pre-trained DenseNet and identifies bias and application-domain boundaries for future work.

V. SUMMARY AND CONCLUSIONS

  • Image representation is especially challenging in digital pathology because WSIs combine complex textures, polymorphism, and very large image sizes.Source: §V · p. 10
  • KimiaNet uses clustering-based, high-cellularity mosaics to train with WSI-level labels when pixel-level annotations are unavailable.Source: §V · p. 10
  • The study trains a densely connected network on 1000 × 1000-pixel patches at 20× magnification from TCGA using weak labels.Source: §V · p. 10
  • Fine-tuning a deep network on sufficiently many histopathology images delivered better performance than a pre-trained network model.Source: §V · p. 10
  • The high-cellularity mosaic facilitated training but may bias KimiaNet toward certain histologic features at the expense of visual similarities.Future work must define the histopathology applications that benefit from KimiaNet image representation.Source: §V · p. 10

VI. APPENDIX

The appendix lists all 32 TCGA primary-diagnosis codes and the corresponding number of evidently diagnosed patients.

VI. APPENDIX

  • Table VI presents the 32 TCGA primary-diagnosis codes in alphabetical order with corresponding patient counts.TCGA denotes The Cancer Genome Atlas.Source: Table VI · p. 12

Related papers

Loading 2101.07903v1…