Source-linked AI summary
Capsule Networks for Brain Tumor Classification based on MRI Images and Course Tumor Boundaries
Parnian Afshar, Konstantinos N. Plataniotis, Arash Mohammadi
TL;DR
Brain tumor classification from MRI is difficult because CNNs do not fully exploit spatial relations and CapsNets can be distracted by image background. The paper modifies CapsNet by adding coarse tumor boundaries alongside MRI inputs, and reports higher classification accuracy than previous CapsNets and CNNs while avoiding exact tumor annotation.
Problem
MRI-based brain tumor classification is challenging, while CNNs inadequately use spatial relations and CapsNets can be sensitive to miscellaneous image background and surrounding tissues.
Method
A modified CapsNet combines raw brain MRI images with a coarse tumor boundary vector before fully connected layers and the final Softmax classifier.
Results
The proposed approach increases classification accuracy compared with previous CapsNets and CNNs and achieves its best performance using brain images with coarse tumor boundaries.
Takeaways & Limitations
The approach avoids exact tumor annotation while helping CapsNet focus on the tumor and consider its relation with surrounding tissues.
Takeaways & Limitations
CNN pooling loses exact feature-detector location, leaving robustness to rotation and other transformations unresolved without impractical coverage of all training situations.
Abstract
from arXiv · showhide
According to official statistics, cancer is considered as the second leading cause of human fatalities. Among different types of cancer, brain tumor is seen as one of the deadliest forms due to its aggressive nature, heterogeneous characteristics, and low relative survival rate. Determining the type of brain tumor has significant impact on the treatment choice and patient's survival. Human-centered diagnosis is typically error-prone and unreliable resulting in a recent surge of interest to automatize this process using convolutional neural networks (CNNs). CNNs, however, fail to fully utilize spatial relations, which is particularly harmful for tumor classification, as the relation between the tumor and its surrounding tissue is a critical indicator of the tumor's type. In our recent work, we have incorporated newly developed CapsNets to overcome this shortcoming. CapsNets are, however, highly sensitive to the miscellaneous image background. The paper addresses this gap. The main contribution is to equip CapsNet with access to the tumor surrounding tissues, without distracting it from the main target. A modified CapsNet architecture is, therefore, proposed for brain tumor classification, which takes the tumor coarse boundaries as extra inputs within its pipeline to increase the CapsNet's focus. The proposed approach noticeably outperforms its counterparts.
1. INTRODUCTION
Brain tumor classification from MRI is clinically important but difficult and dependent on radiologist experience. The paper motivates CapsNets with tumor boundary information to focus on relevant regions while retaining surrounding-tissue cues.
- Clinical motivation: Brain tumors are especially deadly because of aggressive, heterogeneous disease characteristics and low relative survival, making early and accurate typing important.The passage gives an approximate 35% relative survival rate after diagnosis of a primary malignant brain tumor in the US.
- Clinical motivation: MRI offers high-resolution brain-tissue images, but tumor-type recognition remains challenging, error-prone, and time-consuming because it depends heavily on radiologist experience.The passage also notes that images may lack visible landmarks supporting accurate decisions.
- Prior approaches: CNNs enable end-to-end learning without requiring tumor segmentation, but they do not adequately preserve spatial relations and can lack robustness to rotation and affine transformations.The text presents CapsNets as an alternative equipped with Routing by Agreement to improve transformation robustness.
- Paper contribution: The proposed approach feeds coarse tumor boundaries into a CapsNet alongside image information, helping it use surrounding tissue without being distracted from the main target.The boundary information is introduced before fully connected layers and the final Softmax decision layer.
2. PROBLEM FORMULATION
This section introduces CNNs and CapsNets, emphasizing how pooling can discard spatial information and how capsule routing preserves relationships between features.
- 2.1. Convolutional Neural Networks: CNNs combine convolutional, pooling, and sometimes fully connected layers, using shared weights to reduce computational cost while extracting local features.They can learn elementary and higher-order features without prior knowledge of feature types.
- 2.1. Convolutional Neural Networks: A CNN layer transforms the previous layer’s output through convolution with learned kernels followed by an activation function.The kernel size, weights, pre-activation output, and activation function define the layer operation.
- 2.1. Convolutional Neural Networks: Pooling reduces parameters and promotes translation invariance but discards exact feature locations, limiting recognition under rotation and other transformations.Addressing these transformations would require training on all possible situations, which is impractical.
- 2.2. Capsule Networks (CapsNets): CapsNets represent objects with capsules whose vector lengths encode existence probabilities and whose components capture instantiation parameters such as rotation and size.A squashing function is applied to keep capsule-vector lengths below one.
- 2.2. Capsule Networks (CapsNets): CapsNets replace pooling with routing by agreement, allowing lower-level capsules to contribute according to how accurately they predict parent-capsule outputs.The parent considers lower-level capsules that provide correct predictions.
- 2.2. Capsule Networks (CapsNets): During routing, capsule-parent coupling is updated from the agreement between a lower capsule’s prediction and the parent capsule’s actual output.The agreement can be computed using the inner product of the underlying vectors, while coupling coefficients are set by softmax.
- 2.2. Capsule Networks (CapsNets): Each output capsule has a loss that penalizes large instantiation vectors for absent classes, and the total capsule loss sums across output capsules.The class indicator T_j equals 1 when class j is present and 0 otherwise.
- 2.2. Capsule Networks (CapsNets): CapsNets also use a decoder that reconstructs the original image from true-class instantiation parameters, with a smaller loss weight to avoid distracting classification.This decoder is intended to encourage representative features while keeping classification as the main target.
3. THE PROPOSED FRAMEWORK
The proposed framework augments CapsNet with coarse tumor boundaries so it can use surrounding-tissue information while focusing on the tumor region. The architecture combines capsule outputs and boundary information before final classification.
- Tumor location and its relation to surrounding tissues are important for brain tumor classification, but CNN pooling can lose this spatial information.
- CapsNets preserve spatial relations but can be sensitive to miscellaneous image background and account for irrelevant image details.
- The proposed CapsNet receives brain MRI images and coarse tumor boundaries to focus on the main target without requiring detailed tumor annotation.
- The tumor boundary box is concatenated with the masked output of the final capsule layer and passed through fully connected layers before Softmax classification.
- The final loss combines capsule loss with cross-entropy loss, weighting the latter with a smaller γ so it does not dominate the objective.
4. EXPERIMENTAL SETUP
The experiments evaluate the proposed CapsNet on a three-class brain MRI dataset and compare inputs using brain images, segmented tumors, or coarse boundaries. The reported comparison shows the proposed configuration performs best among the evaluated alternatives.
- The proposed CapsNet is compared with scenarios using either brain images or segmented tumor images, as well as CNN-based alternatives.
- The modified CNN uses both brain images and bounding-box coordinates, concatenating the coordinates with its last fully connected layer.
- The proposed CapsNet outperforms CNN in all evaluated situations and achieves its best performance with brain images and coarse tumor boundaries.
5. CONCLUSION
The paper presents a CapsNet that combines raw MRI images with coarse tumor boundaries for brain tumor classification. The authors report increased classification accuracy compared with previous CapsNets and CNNs.
- The proposed architecture incorporates raw MRI brain images and coarse tumor boundaries for tumor classification.
- The approach eliminates the need for exact tumor annotation while helping CapsNet focus on the main area and consider surrounding tissues.
- The authors report increased classification accuracy compared with previous CapsNets and CNNs.
- Future work will investigate CapsNet interpretability for brain tumor classification using output instantiation parameters.