PaperlayerView exact revision on arXiv

Source-linked AI summary

Novel Deep Learning Architectures for Classification and Segmentation of Brain Tumors from MRI Images

Sayan Das, Arghadip Biswas

arXiv:2512.06531v1cs.CVcs.AIcs.LG

TL;DR

Early brain-tumor detection matters for diagnosis and treatment, while prior studies are described as having generalization or test-performance drawbacks. The paper proposes SAETCN for classification and SAS-Net for segmentation, reporting classification accuracies across three datasets and 99.23% overall pixel accuracy for segmentation.

  • Problem

    The paper identifies prior classification studies with limited generalization or poor test performance, against the need for accurate early brain-tumor diagnosis and treatment.

    Source: §1 · p. 2, §2.1 · p. 3

  • Method

    The paper proposes SAETCN for classifying brain tumors and SAS-Net for segmenting tumor regions.

    Source: §1 · p. 2

  • Results

    SAETCN achieved accuracy of 99.38%, 98.69%, and 98.57% on datasets 1, 2, and 3, respectively; SAS-Net achieved 99.23% overall pixel accuracy.

    Source: §1 · p. 2, §4.6 · p. 15

  • Takeaways & Limitations

    SAETCN is also described as predicting whether a brain has any tumor or not.

    Source: §6 · p. 16

    The authors warn that the model may overfit the three datasets used and say larger datasets are needed before real-world hospital use.

    Source: §6 · p. 17

Abstract

from arXiv · show

Brain tumors pose a significant threat to human life, therefore it is very much necessary to detect them accurately in the early stages for better diagnosis and treatment. Brain tumors can be detected by the radiologist manually from the MRI scan images of the patients. However, the incidence of brain tumors has risen amongst children and adolescents in recent years, resulting in a substantial volume of data, as a result, it is time-consuming and difficult to detect manually. With the emergence of Artificial intelligence in the modern world and its vast application in the medical field, we can make an approach to the CAD (Computer Aided Diagnosis) system for the early detection of Brain tumors automatically. All the existing models for this task are not completely generalized and perform poorly on the validation data. So, we have proposed two novel Deep Learning Architectures - (a) SAETCN (Self-Attention Enhancement Tumor Classification Network) for the classification of different kinds of brain tumors. We have achieved an accuracy of 99.38% on the validation dataset making it one of the few Novel Deep learning-based architecture that is capable of detecting brain tumors accurately. We have trained the model on the dataset, which contains images of 3 types of tumors (glioma, meningioma, and pituitary tumors) and non-tumor cases. and (b) SAS-Net (Self-Attentive Segmentation Network) for the accurate segmentation of brain tumors. We have achieved an overall pixel accuracy of 99.23%.

1. Introduction

The paper motivates automated brain-tumor analysis by the importance of early diagnosis and proposes separate deep-learning architectures for classification and segmentation. It reports SAS-Net segmentation accuracy and describes SAETCN’s classification scope.

1. Introduction

  • Accurate early classification and segmentation are presented as important because brain tumors pose a health challenge involving high mortality and complex treatment.The introduction names gliomas, meningiomas, and pituitary tumors as the types addressed.Source: §1 · pp. 1–2
  • The introduction describes converting Figshare MAT files into images and applying contrast enhancement and normalization to the other datasets.It also states that both three- and four-class classification were performed.Source: §1 · p. 2
  • SAETCN is proposed to classify glioma, meningioma, and pituitary tumors from MRI images.The paper describes accurate early classification and segmentation as important for effective treatment.Source: §1 · p. 2
  • 99.23% overall pixel accuracy is reported for SAS-Net, the proposed architecture for segmenting tumor regions.The paper presents segmentation as a distinct contribution alongside tumor classification.Source: §1 · p. 2

2. Related Works

Prior work spans deep-learning approaches to brain-tumor detection, classification, and segmentation, but the review identifies limitations in classification-model generalization and performance on test or original data. Segmentation research includes reviews, network proposals, and comparisons of deep-learning methods.

2.1. Review of previous works on classification

  • Classification methods reviewed include CNNs, autoencoders, capsule networks, and other machine-learning approaches for distinguishing brain-tumor types.Reported results include Saeedi et al.'s 93.44% test accuracy for a 2D CNN and Abiwinanda's 84.19% validation accuracy.Source: §2.1 · pp. 2–3
  • Earlier classification studies report limited generalization, weaker test performance, or better results on augmented than original data.The review notes potential artifacts and overfitting associated with augmentation.Source: §2.1 · p. 3

2.2. Review of previous works on segmentation

  • Segmentation literature covers surveys, improved residual networks, and comparisons that report deep-learning methods, particularly U-Net variants, outperforming other approaches.The cited work also reviews segmentation under imbalanced conditions and across imaging modalities.Source: §2.2 · pp. 3–4

3.1. Overall Framework

The paper presents a framework for both brain-tumor classification and segmentation. SAETCN distinguishes tumor types using self-attention, while SAS-Net uses similar techniques to delineate tumor boundaries.

3.1. Overall Framework

  • The framework pairs SAETCN for distinguishing tumor types with SAS-Net for delineating tumor boundaries using self-attentive techniques.The paper describes the two networks as a framework addressing classification and segmentation.Source: §3.1 · p. 4

3.2. Classification Architecture

SAETCN classifies tumor and non-tumor MRI images through serial self-attention blocks, with feature extraction modules followed by adaptive average pooling and a dense layer. Its design combines multi-branch feature extraction with a skip connection.

3.2. Classification Architecture

  • 99.31% and 98.20% accuracy were achieved on the Brain MRI Kaggle and Jun Cheng Figshare datasets, respectively.The authors report that SAETCN outperformed many state-of-the-art deep-learning architectures.Source: §3.2 · p. 4

3.2.1. SAETCN Architecture

  • SAETCN classifies three tumor classes and non-tumor cases through 16 serial Self Attention Enhancement Blocks, followed by adaptive average pooling and a dense layer.Its named components include NCAB, Initial TriSAE, QuadSAE, HexaSAE, and Final SAE-Fusion.Source: §3.2.1 · pp. 4–5

3.2.2. Normalised Convolutional Activation Block

  • NCAB applies a 7×7 convolution, ReLU, batch normalization, and max pooling to extract initial spatial features and reduce dimensionality.The authors describe this block as detecting important textures and edges.Source: §3.2.2 · p. 5

3.2.3. Self Attention Enhancement Block

  • SAEB sends the input through four branches with different convolution or pooling operations, concatenates and batch-normalizes their outputs, then adds a skip-path output and applies ReLU.The branches include 1×1, 3×3, and 5×5 convolutional paths and a max-pooling path.Source: §3.2.3 · pp. 5–6

3.2.4. Initial TriSAE Module

The Initial TriSAE and QuadSAE modules stack Self Attention Enhancement Blocks in sequence, increasing the channel count from 64 to 256 and then to 512.

3.2.4. Initial TriSAE Module

  • The Initial TriSAE Module passes NCAB features through three serial SAEBs, mapping 64 input channels to 256 output channels.Its output is denoted FSAEB3.Source: §3.2.4 · p. 6

3.2.5. QuadSAE Module

  • The QuadSAE Module passes the Initial TriSAE output through four serial SAEBs, mapping 256 input channels to 512 output channels.Its output is denoted FSAEB7.Source: §3.2.5 · p. 6

3.2.6. HexaSAE Module

The classification architecture extends HexaSAE into a final fusion module, then uses pooled features and dense layers to classify tumor and non-tumor MRI images. Its design uses 16 SAE blocks across four modules, with parallel and skip connections described as supporting feature extraction.

3.2.6. HexaSAE Module

  • HexaSAE applies six SAE operations to a 512-channel input and produces 1024 output channels.It uses the QuadSAE output as its input.Source: §3.2.6 · p. 7

3.2.7. Final SAE Fusion Module

  • The Final SAE Fusion Module takes the HexaSAE output through three SAE blocks, expanding the representation from 1024 to 2048 channels.Average pooling and flattening precede a 2048-neuron ReLU dense layer and a four-neuron softmax output for three tumor classes and non-tumor cases.Source: §3.2.7 · p. 7

3.2.8. Model Complexity Analysis

  • The Self Attention Enhancement Block uses four parallel connections and a skip connection; the authors attribute multiple feature extraction and faster convergence to residual and inception operations.The paper describes the paths as capturing low-level features, nonlinear features, pooled features, and residual information.Source: §3.2.8 · p. 8
  • The authors describe four modules as extracting outline, deeper pixel-level, and tumor-class features, with the fourth intended to improve performance.The fourth module is described as performance-oriented, while the first three are assigned distinct feature-extraction roles.Source: §3.2.8 · p. 8
  • The authors select 16 serial SAE blocks, divided into four modules, because their experiments found classification performance saturated beyond 16 blocks.They describe the modules as having different feature-extraction capabilities.Source: §3.2.8 · pp. 8–9

3.3. Segmentation Architecture

SAS-Net segments brain tumors using five sequential Self Attention Enhancement Blocks and a decoder that combines their features. Its SFD blocks upsample and integrate encoder features through residual, multi-scale processing.

3.3. Segmentation Architecture

  • 99.23% pixel accuracy is reported for the proposed SAS-Net tumor segmentation architecture.The paper links tumor localization to diagnosis planning.Source: §3.3 · p. 9

3.3.1. Self-Attentive Segmentation Network

  • SAS-Net connects five sequential SAE blocks to a decoder containing four Segmental Feature Decoding Blocks, then applies a convolutional layer for segmentation.Each SAE block passes output to the next encoder block and to a corresponding decoder block.Source: §3.3.1 · p. 9

3.3.2. Segmental Feature Decoding Block

  • Each SFD block upsamples its input, concatenates corresponding encoder features, and processes them with parallel convolutional paths and residual connections.The paths use 1×1, 3×3, and 5×5 convolutions, plus max pooling followed by a 1×1 convolution.Source: §3.3.2 · pp. 9–10

3.3.3. Enhanced Segmentation Integration Module

  • The Enhanced Segmentation Integration Module is a decoder of four sequential SFD blocks, each combining the previous decoder output with a corresponding SAE block.The first SFD block instead takes inputs from SAE blocks 5 and 4; the architecture uses five SAE blocks.Source: §3.3.3 · p. 10

4. Experiment and Result

The experiments evaluate SAETCN classification on three MRI datasets and SAS-Net segmentation on BraTS2020. The authors report higher classification accuracies than several named models and strong segmentation metrics relative to prior work.

4.1. Dataset and Preprocessing

  • The classification datasets comprise a 7,023-image four-class Brain MRI dataset, 3,064 Figshare images across three tumor classes, and a 4,549-image four-class custom cross-mixed dataset.The paper describes the third dataset as a mix of the first two and frames it as a test of generalization capability.Source: §4.1 · p. 11

4.2. Experimental Setup

  • Images are resized and preprocessed; classification training uses a 0.0001 learning rate, Adam, and cross-entropy loss, while segmentation uses Binary Cross Entropy with Logits Loss.The paper states that datasets are split into training and test sets at 80:20.Source: §4.1 · p. 11, §4.2 · p. 11

4.3. Evaluation Metrics for Classification

  • Classification evaluation uses micro-averaged metrics and ROC-AUC calculated with one-vs-one and micro-averaged approaches.Reported metrics include precision, F1 score, R2 score, recall, confusion matrices, and ROC-AUC.Source: §4.3 · pp. 12–13

4.4. Evaluation Metrics for Segmentation

  • Segmentation evaluation includes IoU, MCC, DSC, specificity, and Boundary F1, covering overlap, classification balance, and boundary accuracy.The paper describes MCC as useful even when class sizes differ.Source: §4.4 · p. 13

4.6. Comparison of our Classification Model with Previous Works

  • 99.38%, 98.69%, and 98.57% accuracy are reported for Dataset 1, Dataset 2, and Dataset 3, respectively, beating the compared state-of-the-art models.The compared models include EfficientNetB4, ResNet18, InceptionNetV3, Swin Transformer, and ViT.Source: §4.6 · p. 15, Table 11 · p. 16

4.7. Comparison of our segmentation model with previous works

  • SAS-Net reports 99.79% DSC, 93.55% specificity, and 99.89% sensitivity, outperforming prior models across most metrics.Wentau Wu et al. report higher specificity at 99.82%, while the paper says their DSC and sensitivity are lower than SAS-Net’s.Source: §4.7 · p. 16, Table 12 · p. 16

5. Abalation Study

The ablation study identifies 16 SAE blocks as the best-performing configuration among the tested combinations.

5. Abalation Study

  • A module-wise ablation study was performed, with results reported in Table 10.The supplied passage does not state the individual module ablation values.Source: §5 · p. 16
  • The authors report that choosing 16 SAEB blocks gives better classification performance among every combination tested, while Figure 6 provides a detailed Grad-CAM view of all model components.Figure 16 presents evaluation metrics for different model components.Source: §5 · p. 16, Figure 16 · p. 16

6. Conclusions

The conclusion presents SAETCN for classifying tumor types and detecting tumor presence, with high reported accuracy. It also flags possible overfitting and describes broader generalization and mobile use as future work.

6. Conclusions

  • SAETCN classifies glioma, meningioma, and pituitary tumors and predicts whether a brain has a tumor.The authors describe mobile or web integration as a way to make the model easier to use, including in remote areas.Source: §6 · p. 16
  • 99.31% and 99.20% accuracy were reported in the dataset comparison, where the authors say SAETCN beat most compared models.The authors caution that the model may be overfit on the three datasets and should be trained on larger datasets before real-world hospital use.Source: §6 · p. 17
  • The authors identify improved real-time generalization as future work and propose training with larger data and fine-tuning on MRI-machine-specific data.They also plan a mobile application for doctors and patients.Source: §6 · p. 17

7. Data Availablity

The paper makes its code and saved model architectures publicly available and links the datasets from the repository, while the custom dataset is available on request. It also reports that the architectures are integrated into the NeuroGuard Web Application.

7. Data Availablity

  • The computational code and saved model architectures are publicly available in the full GitHub repository.The passage provides the repository address.Source: §7 · p. 17
  • The two public datasets and custom dataset are linked in the repository, with the custom dataset available upon reasonable request.Source: §7 · p. 17
  • The SAETCN and SASNET architectures are integrated into the NeuroGuard Web Application backend for brain-tumor classification and detection from MRI images.Source: §7 · p. 17
Loading 2512.06531v1…