Source-linked AI summary
Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations
H. Fabelo, S. Ortega, D. Ravi, B. R. Kiran, C. Sosa, D. Bulters, G. M. Callico, H. Bulstrode, A. Szolna, J. F. Pineiro, S. Kabwama, D. Madronal, R. Lazcano, A. J. OShanahan, S. Bisshopp, M. Hernandez, A. Baez-Quevedo, G. Z. Yang, B. Stanciulescu, R. Salvador, E. Juarez, R. Sarmiento
TL;DR
Brain tumor margins are difficult to identify during surgery because infiltrative tumors resemble surrounding normal brain, while existing delineation tools have limitations. The study develops a hybrid spatial-spectral classification framework combining supervised and unsupervised learning for intra-operative HSI. Preliminary evaluation on five patients produced classification maps that specialists judged promising for accurate tumor-area delineation, with results accelerated to surgical time.
Problem
Infiltrative brain tumors are difficult to distinguish from surrounding normal tissue during surgery, risking residual tumor or excessive normal-tissue removal.
Method
The study combines supervised spatial-spectral classification with unsupervised clustering and majority-vote fusion to analyze intra-operative hyperspectral images.
Results
Overall accuracy exceeded 99% for discriminating normal tissue, tumor tissue, blood vessels, and background in specialist-labeled tissue classification.
Takeaways & Limitations
The resulting system provides promising tumor-region identification and can produce classification results in approximately 1 minute during neurosurgical operations.
Takeaways & Limitations
The study reports false positives, tissue misclassifications, and a need for clinical validation and further algorithm acceleration.
Abstract
from arXiv · showhide
Surgery for brain cancer is a major problem in neurosurgery. The diffuse infiltration into the surrounding normal brain by these tumors makes their accurate identification by the naked eye difficult. Since surgery is the common treatment for brain cancer, an accurate radical resection of the tumor leads to improved survival rates for patients. However, the identification of the tumor boundaries during surgery is challenging. Hyperspectral imaging is a noncontact, non-ionizing and non-invasive technique suitable for medical diagnosis. This study presents the development of a novel classification method taking into account the spatial and spectral characteristics of the hyperspectral images to help neurosurgeons to accurately determine the tumor boundaries in surgical-time during the resection, avoiding excessive excision of normal tissue or unintentionally leaving residual tumor. The algorithm proposed in this study to approach an efficient solution consists of a hybrid framework that combines both supervised and unsupervised machine learning methods. To evaluate the proposed approach, five hyperspectral images of surface of the brain affected by glioblastoma tumor in vivo from five different patients have been used. The final classification maps obtained have been analyzed and validated by specialists. These preliminary results are promising, obtaining an accurate delineation of the tumor area.
Introduction
Brain tumor margins remain difficult to define during surgery because tumors infiltrate surrounding normal tissue and existing tools have limitations. This pilot study investigates intra-operative hyperspectral imaging as a potential approach for identifying and delineating brain tumors.
- Introduction: Tumor infiltration makes it difficult for surgeons to distinguish tumor from normal brain tissue by naked-eye inspection.This can result in residual tumor or excessive removal of normal tissue.
- Introduction: Hyperspectral imaging captures spatial and spectral information beyond the three electromagnetic bands visible to the human eye.Each pixel contains spectral information that correlates with the chemical composition of the captured scene.
- Introduction: HSI is presented as a non-invasive technique that can rapidly acquire and analyze tissue information for medical diagnosis.The study frames HSI as promising for intra-operative tumor detection.
- Introduction: The pilot study investigates whether intra-operative HSI can identify and delineate brain tumors.The work is conducted within the European HELICoiD collaborative project.
- Introduction: The HELICoiD demonstrator uses a pushbroom hyperspectral camera covering 400–1000 nm with 826 spectral bands and 1004 spatial pixels.Its scanning platform supplies the second spatial dimension required by the pushbroom camera.
In-vivo human brain hyperspectral image database
The study database comprises five in-vivo hyperspectral brain-surface images from adults with histopathologically confirmed grade IV glioblastoma. Tissue labels were supported by surgical knowledge, biopsy pathology, and specialist-assisted spectral labeling.
- In-vivo human brain hyperspectral image database: Five in-vivo brain-surface hyperspectral images were collected from five adults undergoing craniotomy for intra-axial brain-tumor resection.Images came from hospitals in Spain and the United Kingdom, and all patients had grade IV glioblastoma confirmed by histopathology.
- In-vivo human brain hyperspectral image database: Biopsies from marked or clearly identified tumor areas were sent for pathological confirmation of tumor presence or absence.The resulting histopathological information contributed to the supervised-learning labels.
- In-vivo human brain hyperspectral image database: The labeling workflow used Spectral Angle Mapper to compare image-pixel spectra with reference spectra and generate maps for four classes.The classes were normal tissue, tumor tissue, blood vessel, and background.
- In-vivo human brain hyperspectral image database: Gold-standard reliability combined neuronavigation-based marker placement, operating-surgeon expertise, and pathological analysis of resected tissue.These sources supported labeling of normal tissue, blood vessels, background, and tumor tissue.
Brain cancer detection algorithm
The proposed framework combines preprocessing, dimensionality reduction, spatial-spectral supervised classification, unsupervised segmentation, and hybrid classification to analyze brain-surface hyperspectral images. It uses SVMs, FR-t-SNE, KNN filtering, and cluster-based merging to produce a final tumor-delimitation map.
- The classification framework uses five stages: preprocessing, dimensionality reduction, spatial-spectral supervised classification, unsupervised clustering segmentation, and hybrid classification.
- The final hybrid map combines supervised tissue labels with unsupervised segmentation, using spatial clusters to refine classification while retaining semantic labels.
- Preprocessing homogenizes pixel spectra and reduces hyperspectral dimensionality before subsequent characterization and classification.
- FR-t-SNE provides a consistent low-dimensional representation to address the instability of conventional t-SNE embeddings across images.
- SVMs are trained with labeled in-vivo brain samples, while spatial homogenization incorporates neighborhood information through KNN filtering of supervised confidence maps.
- Increasing λ or K can oversmooth classification maps and reduce accuracy, whereas selected values produce homogenized maps without oversmoothing.
Brain cancer detection algorithm acceleration
A preliminary demonstrator accelerates parts of the brain-cancer detection application using a computer and an MPPA-256-N hardware accelerator.
- The demonstrator uses a computer and an MPPA-256-N accelerator containing 256 processing units.
Evaluation metrics
The study evaluates supervised classifiers with cross-validation using sensitivity, specificity, and overall accuracy, then assesses whole-cube classification with neurosurgical review.
- A 10-fold cross-validation procedure measures supervised-model performance on labeled information.
- The metrics are computed from true-positive, false-positive, true-negative, and false-negative outcomes.
- Sensitivity measures correctly identified actual positives, while specificity measures correctly identified actual negatives.
- Overall accuracy measures the model’s ability to correctly predict labels for new or previously unseen data.
- After quantitative evaluation, the trained SVM classifies the whole hyperspectral cube and neurosurgeons assess its tissue-discrimination quality.
Hyperspectral imaging can distinguish between tumor and normal tissue pixels by their spectra
Hyperspectral signatures show strong differences between normal and tumor tissue, although tumor and blood-vessel spectra can overlap. Supervised SVM classification therefore separates labeled classes accurately but can produce blood-related false positives.
- Tumor, normal, and blood-vessel spectral signatures vary with tissue heterogeneity, particularly within the tumor class.
- Differences between normal and tumor spectra support successful supervised classification of these labeled pixels.
- Linear SVM achieved more than 99% overall accuracy with lower computational cost than the tested alternative kernels.
- Specificity and sensitivity exceeded 96%, reaching 100% in most patient-and-class evaluations with the linear kernel.
- Supervised maps identified tumor areas but included false positives caused by spectral similarities between tumor tissue, blood vessels, and extravasated blood.
Improving the spatial coherence of the supervised classification maps
The supervised maps are spatially refined by combining FR-t-SNE one-band representations with classification probabilities and KNN filtering. This homogenizes class regions and supplies contextual information for later cancer-map generation.
- Improving spatial coherence: FR-t-SNE generates one-band hyperspectral representations that reveal significant image information with high contrast.The representations highlight tumor borders and help identify tumor regions in each image.
- Improving spatial coherence: KNN filtering uses FR-t-SNE representations and supervised classification probability scores as inputs.The combined inputs provide contextual information for each pixel in the classification scheme.
- Improving spatial coherence: The resulting spatially optimized maps homogenize each class region and increase coherence across the classification maps.The visual differences from the original supervised maps are not very noticeable, but the homogenization supports the final detection stage.
Unsupervised clustering for accurate boundaries delineation of the brain surface
HKM clustering delineates tissue structures and brain-surface regions, but its clusters lack class identities. Therefore, the unsupervised maps must be combined with supervised class information.
- Unsupervised clustering: HKM clustering delineates blood vessels, ring markers, tissue regions, and the parenchymal brain area across five patient images.Within the parenchymal area, tissue structures and region boundaries are highlighted with high accuracy.
- Unsupervised clustering: The clustering maps accurately separate tissue structures but do not identify which class each randomly colored cluster represents.The colors are randomly selected, so the segmentation output alone is not meaningful for cancer classification.
- Unsupervised clustering: Supervised identified classes must therefore be combined with the unsupervised clusters to produce interpretable cancer maps.This combination links accurate cluster boundaries with tissue-class labels.
Delimiting and identifying the human brain area affected by cancer
The final detection stage combines clustered regions with spatially homogenized supervised maps to assign tissue classes and represent mixed-class uncertainty. Across five patient images, the processing setup was assessed for surgical-time applicability.
- Final classification: The MV algorithm assigns the maximum majority supervised class to each HKM segmentation cluster, producing final classification maps with better-delineated class boundaries.The maps can reduce tumor regions or reveal mixed normal and tumor tissue within areas previously labeled only as tumor.
- Mixed-tissue representation: OMD maps degrade class colors according to MV probabilities, signaling clusters where multiple tissue classes may be merged.For example, an 80% tumor probability produces a 20% color degradation for the tumor class.
- Mixed-tissue representation: TMD maps mix the colors of the three highest MV probabilities to represent classes that are mixed within a cluster.This extends the maximum-probability representation by retaining information from three leading class probabilities.
- Processing time: Processing time was evaluated on five patient images using sequential CPU execution and hardware acceleration for the spatial-spectral supervised-classification stage.The accelerated implementation partitions unsupervised clustering to the CPU and supervised classification to hardware, while accounting for transmission time.
- Processing time: The accelerated total processing time is determined by the longer of the hardware supervised-classification stage and CPU unsupervised-clustering stage.Without acceleration, spatial-spectral supervised classification is the most time-consuming stage.
Discussion
The study addresses difficult intraoperative tumor-margin identification with a hybrid hyperspectral classification framework that combines spatial-spectral supervised classification and unsupervised segmentation. Preliminary analyses show accurate tissue discrimination, tumor-region identification, and surgical-time classification, while boundary delineation is supported by the fused maps.
- Discussion: Hyperspectral imaging addresses the difficulty of distinguishing infiltrative brain tumors from surrounding normal tissue during surgery.The technique is non-contact, non-ionizing, and non-invasive.
- Discussion: The methodology combines medical expertise with an algorithm exploiting spatial and spectral features of hyperspectral images.Medical doctors generated and labeled the image database, while engineers developed the digital processing and detection algorithm.
- Discussion: Overall accuracy exceeded 99% for supervised discrimination of normal tissue, tumor tissue, blood vessels, and background.The supervised models were generated from tissues previously labeled by specialists.
- Discussion: The fused spatial-spectral supervised and unsupervised maps identify tumor regions, delineate tissue boundaries, and assign identifiable tissue classes to segmentation clusters.The supervised maps locate tumors but do not alone provide accurate boundary delineation; fusion with HKM-based segmentation supplies the boundary information.
- Discussion: The TMD representation conveys mixed class composition within clusters and helps assess tumor infiltration into surrounding normal brain when judging resection extent.The MV and OMD maps provide alternative class-probability representations, while TMD represents mixtures among classes.
- Discussion: Classification results were obtained in approximately 1 minute, with an average 6.43x speedup over a sequential CPU implementation.The reported timing is shorter than the more-than-30-minute duration stated for intraoperative pathological analysis or intra-surgical magnetic resonance.
Limitations
The study reports false positives, tissue misclassifications, and unresolved validation and acceleration requirements. These limitations constrain confidence in boundary accuracy and indicate that broader clinical and computational evaluation remains necessary.
- Limitations: False positives were observed, including confusion between blood or blood vessels and tumor tissue caused by high vascularized-tissue variability.The authors state that these false positives did not affect the identified tumor area, leaving tumor margins clearly evident.
- Limitations: Extensive clinical validation is required to determine whether tumor boundaries represented in the TMD map are accurately identified.The proposed validation requires several tumor-area boundary biopsies analyzed by pathologists.
- Limitations: Misclassifications occurred among highly vascularized tissues, including extravasated blood and edematous normal tissue classified as blood vessel or hypervascularized tissue.The authors attribute this issue to similar spectral characteristics and suggest incorporating those differences into training or creating a separate class.
- Limitations: Further acceleration is needed because the current algorithm produces results in surgical time but not yet in real time.The authors suggest heterogeneous high-performance computing could produce the final TMD map in less than one second.
Conclusions
The study develops a surgical-time brain-tumor classification algorithm intended to support accurate boundary identification while limiting excessive normal-tissue resection and residual tumor. Further work targets generalization, algorithm optimization, validation, larger image databases, and hardware acceleration.
- The algorithm classifies hyperspectral brain-tumor images during neurosurgical operations to support tumor-boundary identification.
- Accurate boundary identification is intended to avoid excessive normal-brain resection and unintentionally leaving residual tumor.
- Ongoing investigations address result generalization, algorithm optimization, validation, database expansion, and acquisition-system improvement.
- GPU and FPGA platforms are being considered to balance real-time execution, memory usage, and power dissipation in heterogeneous implementations.
- Supplementary evaluations report SVM confusion matrices using linear, polynomial, RBF, and sigmoid kernels with 10-fold cross-validation for each patient.