Source-linked AI summary
In-Vivo Hyperspectral Human Brain Image Database for Brain Cancer Detection
H. Fabelo, S. Ortega, A. Szolna, D. Bulters, J. F. Pineiro, S. Kabwama, A. Shanahan, H. Bulstrode, S. Bisshopp, B. R. Kiran, D. Ravi, R. Lazcano, D. Madronal, C. Sosa, C. Espino, M. Marquez, M. De la Luz Plaza, R. Camacho, D. Carrera, M. Hernandez, G. M. Callico, J. Morera, B. Stanciulescu, G. Z. Yang, R. Salvador, E. Juarez, C. Sanz, R. Sarmiento
TL;DR
Medical hyperspectral research lacks specific public datasets, limiting study of brain-tumor delineation with this modality. The paper constructs and evaluates an in-vivo human brain hyperspectral database using a customized acquisition system and specialist labeling, while documenting repeatability and scope constraints. The resulting resource is publicly accessible and supports four-class analysis of brain tissue, vessels, and background.
Problem
Specific, publicly available hyperspectral medical data are lacking for studying brain-tumor delineation and identification during surgery.
Method
The paper builds an in-vivo brain hyperspectral database using customized acquisition, repeatability assessment, and semi-automatic Spectral Angle Mapper labeling.
Results
The database contains 36 images from 22 patients, with four labeled classes and public access to the hyperspectral data.
Takeaways & Limitations
The public database supports research on identifying tissue and tumor types, delineating tumor boundaries and other regions, and providing information useful during neurosurgery.
Takeaways & Limitations
Acquisition was limited by in-vivo access and the demonstrator's need to capture tumors that were surface-accessible or sufficiently easy to focus on.
Abstract
from arXiv · showhide
The use of hyperspectral imaging for medical applications is becoming more common in recent years. One of the main obstacles that researchers find when developing hyperspectral algorithms for medical applications is the lack of specific, publicly available, and hyperspectral medical data. The work described in this paper was developed within the framework of the European project HELICoiD (HypErspectraL Imaging Cancer Detection), which had as a main goal the application of hyperspectral imaging to the delineation of brain tumors in real-time during neurosurgical operations. In this paper, the methodology followed to generate the first hyperspectral database of in-vivo human brain tissues is presented. Data was acquired employing a customized hyperspectral acquisition system capable of capturing information in the Visual and Near InfraRed (VNIR) range from 400 to 1000 nm. Repeatability was assessed for the cases where two images of the same scene were captured consecutively. The analysis reveals that the system works more efficiently in the spectral range between 450 and 900 nm. A total of 36 hyperspectral images from 22 different patients were obtained. From these data, more than 300 000 spectral signatures were labeled employing a semi-automatic methodology based on the spectral angle mapper algorithm. Four different classes were defined: normal tissue, tumor tissue, blood vessel, and background elements. All the hyperspectral data has been made available in a public repository.
I. INTRODUCTION
Hyperspectral imaging offers non-invasive, non-ionizing spectral information for medical applications, including brain-tumor analysis. The paper introduces a HELICoiD database and acquisition system designed to support intraoperative tumor delineation.
- Motivation: HSI captures information across many contiguous, narrow spectral bands beyond conventional RGB imaging.Its medical relevance is linked to tissue-specific chemical signatures associated with cancer.
- Motivation: Current intraoperative guidance methods face limitations including inaccurate tumor boundaries from brain shift or demanding MRI-compatible operating environments.Intraoperative MRI also has a lower refreshing rate than HSI.
- Contribution: The paper presents the first in-vivo hyperspectral human brain image database for studying tumor delineation and identification during surgery.The database was developed within the HELICoiD project.
- Contribution: The database supports machine-learning research on discriminating tumor from normal brain tissue during surgical procedures.The project also targeted intraoperative integration of HSI with image-guided surgery.
- Acquisition system: The customized pushbroom system captures VNIR data from 400 to 1000 nm using 826 spectral bands and 1004 spatial pixels per line.A scanning platform supplies the second spatial dimension of the hyperspectral cube.
III. IN-VIVO HS HUMAN BRAIN DATABASE
The database study recruited adult patients undergoing craniotomy for intra-axial brain-tumor resection at hospitals in the UK and Spain. Data collection occurred in two campaigns from March 2015 to June 2016.
- Participants: Adult patients undergoing craniotomy for intra-axial brain-tumor resection were approached at both participating sites.The sites were the University Hospital of Southampton in the UK and University Hospital Doctor Negrin in Spain.
- Data collection: Two acquisition campaigns covered the period from March 2015 to June 2016.
B. HS IMAGE ACQUISITION PROCEDURE
The acquisition procedure captures hyperspectral images of the exposed brain surface during neurosurgery, using image-guided markers to identify tumor and normal tissue locations. Images are acquired after durotomy and before deeper tissue layers are breached when the tumor reaches the surface.
- Procedure: The procedure comprises patient preparation, image acquisition, tissue resection, neuropathology evaluation, and sample labeling.
- Patient preparation: Preoperative CT and MRI scans are uploaded to the image-guided stereotactic system before surgery.
- HS image acquisition: Hyperspectral images are captured after durotomy, before the arachnoid and pia are breached when the tumor extends to the brain surface.
- Marker registration: Sterilized rubber-ring markers identify tumor and normal brain positions, with the image-guided pointer registering marker locations to MRI or CT coordinates.
- HS image acquisition: The acquisition operator images the exposed brain surface after surgeons place normal markers using visual appearance, anatomy, and image-guided feedback.
3) TISSUE RESECTION
Tissue resection links intraoperative hyperspectral observations to neuropathological diagnoses and subsequent labeling. Surgeons collect marked tissue samples, while labeling uses specialist information and spectral references to construct four-class gold-standard maps.
- Tissue resection: Surgeons remove tissue from inside the tumor marker, assign container identifiers, and send samples for neuropathological analysis.
- Tissue resection: A second image set may be acquired during resection when tumor and macroscopically normal brain are exposed and surgery can be safely paused.
- Neuropathology: Neuropathologists use H&E staining and routine clinical techniques to classify samples as tumor or normal brain and subdivide tumors by type and grade.
- Sample labeling: Reference pixels from marked and non-tumor regions guide semi-automatic labeling, while specular-reflection pixels are excluded.
- Gold-standard classes: Gold-standard maps contain four classes: tumor tissue, normal brain tissue, blood vessels, and background.Possible normal inflamed tissue is included in the normal class.
C. DATA PRE-PROCESSING
The database uses calibrated and filtered hyperspectral data, then combines expert-guided reference pixels with SAM-based labeling to create partial gold-standard maps across four tissue classes.
- Preprocessing: Four preprocessing steps calibrate, denoise, average spectral bands, and normalize each hyperspectral image.
- Gold-standard constraints: Gold-standard maps cannot achieve 100% pixel-level certainty for living human brain tissue because complete pathological examination would require resecting exposed tissue.
- Labeling basis: The partial gold standard combines pathology, neuronavigation, and neurosurgeon expertise to identify tumor, normal, hypervascularized, and background regions.
- SAM-based labeling: SAM-based labeling improves reliability by visually checking spectral similarity while reducing the time required to label hyperspectral pixels manually.
- SAM-based labeling: The semi-automatic labeling tool selects reference pixels in synthetic RGB images and propagates labels using spectral-angle similarity masks.
- Threshold selection: Thresholds are selected separately for reference pixels and classes, with greater variability in background because it includes diverse materials and tissues.
V. REPEATABILITY ANALYSIS OF THE HS ACQUISITION SYSTEM
The acquisition system was evaluated through spatial and spectral repeatability experiments using repeated hyperspectral captures of identical scenes.
- Repeatability evaluation: Two repeatability experiment types assessed systematic acquisition errors and consistency of spectra from consecutively captured images of the same scene.
- Repeatability evaluation: The experiments evaluated both spatial repeatability and spectral repeatability of the hyperspectral acquisition system.
- Repeatability evaluation: Repeated captures were performed under controlled conditions to examine whether the system reproduced scene information consistently.
A. REPEATABILITY DATASET
The repeatability dataset contains paired captures of three scenes selected to test uniform, high-contrast, and spectrally complex conditions. Scatterplot behavior varied with scene contrast, while the defined metrics quantify spatial and spectral consistency.
- Dataset composition: Three scene pairs—a white reference tile, chessboard pattern, and book-cover fragment—were captured twice consecutively under identical environmental conditions.
- Dataset composition: The white tile and chessboard provide geometrically simple references, whereas the book-cover fragment contains multiple reflections and changing reflected-light paths.
- Scatterplot analysis: More than 33 million voxel pairs from 200 × 200-pixel sections were compared between repeated hyperspectral cubes.
- Scatterplot analysis: The homogeneous white tile showed low voxel variability, while the high-contrast chessboard produced greater scatter and the book cover showed lower expansion than the chessboard.
- Repeatability metrics: Repeatability error is ideally zero, while lower relative difference indicates better repeatability; signal-to-noise and noise-to-signal ratios provide inverse perspectives.
- Repeatability metrics: The relative-difference metric compares absolute differences with the mean values of the two vectors and is reported as a percentage.
C. SPECTRAL REPEATABILITY
Spectral repeatability evaluates consistency across line scans and CCD bins in pushbroom hyperspectral cubes. Repeatability worsens as scene complexity increases, especially at pattern boundaries and reflective or moving surfaces.
- Acquisition and measures: Pushbroom hyperspectral cubes are assembled from contiguous, nonoverlapping line scans, enabling repeatability analysis along line scans and CCD-bin rows.The CCD sensor records spectral bands across rows while each column captures one pixel spectrum.
- Acquisition and measures: Spectral repeatability experiments measure errors from system vibrations, interline scanning, and differential CCD-bin responses.These are identified as common error sources in pushbroom hyperspectral systems.
- Line-scan repeatability: As scene complexity increases, average S/N decreases and mean RDmean increases across line scans.Homogeneous white-reference imagery remains stable, whereas chessboard and book-cover scenes produce larger repeatability errors.
- Line-scan repeatability: Chessboard boundaries produce prominent line-scan errors, while book-cover errors spread across scans because of surface reflections and system movement.The chessboard has the worst S/N value, although its average RDmean is lower than the book cover’s.
- CCD-bin repeatability: CCD-bin repeatability is better at the center than at the sensor borders for the white reference, while chessboard imagery again has the worst S/N value.Border degradation is attributed to optical aberrations and inhomogeneous illumination across the pushbroom scan direction.
- Spatial interpretation: At λ = 690.78 nm, spatial repeatability maps identify low-repeatability regions concentrated around high-entropy boundaries, letters, drawings, and surface relief.These regions cannot be reproduced accurately because they require fine-resolution spatial scanning.
D. SPATIAL REPEATABILITY
Spatial repeatability compares corresponding pixel values at fixed wavelengths across repeated hyperspectral cubes. It is strongest near central wavelengths and weaker toward spectral-range boundaries, with material composition also affecting the pattern.
- Definition and evaluation: Spatial repeatability measures differences between corresponding pixel values at a fixed wavelength in repeated hyperspectral cubes.The analysis uses S/N and RDmean metrics for white-tile, chessboard, and book-cover cube pairs.
- Spatial maps: At λ = 690.78 nm, false-color spatial maps represent S/N and RDmean across white-tile, chessboard, and book-cover cube pairs.The wavelength was selected as an example of error at a centered spectral position.
- Spectral behavior: Spatial repeatability is better at central wavelengths and worse toward the first and last spectral bands.The reported explanation is stronger absorption of shorter-wavelength photons near the CCD surface rather than the detector’s active region.
- Material effects: Book-cover S/N oscillations reflect multiple materials with low and high reflectance at different wavelengths.This material heterogeneity contributes to the observed spatial repeatability pattern.
- Operational range: The system’s efficient operating range is approximately 450–900 nm, so values outside that interval were excluded during preprocessing.This range was selected from the repeatability experiments.
VI. DATABASE SUMMARY
The labeled gold-standard database contains in-vivo brain-surface hyperspectral images from multiple patients and tumor types. It records tissue classes and patient-level spectral summaries while documenting cases where tumor labeling was not possible.
- Dataset composition: 36 in-vivo brain-surface images from 22 patients compose the gold-standard dataset.The images include primary glioblastoma and anaplastic oligodendroglioma tumors and secondary lung and breast tumors.
- Label classes: The database labels tumor and normal tissue when possible, along with blood vessels and background elements in the surgical scene.Background includes tissues, materials, or substances not relevant to surgical resection.
- Spectral summaries: Figure 11 reports average and standard deviation spectral signatures across patients for normal tissue, blood vessels, and multiple tumor types.Tumor categories include glioblastoma, anaplastic oligodendroglioma, secondary breast, and secondary lung tumors.
- Labeling scope: Some diagnosed tumor images contain no labeled tumor samples because perspective, depth, or related acquisition difficulties prevented labeling.The database records these cases alongside image characteristics and pathological diagnoses.
VII. LIMITATIONS
The database is preliminary because patient availability, acquisition constraints, and incomplete biological-spectral correlations limit its coverage and operational scope. Nevertheless, its labeled data support investigations into tissue classification and tumor delineation.
- Data and patient coverage: During two years, acquisition captured mainly available tumors, particularly glioblastomas, limiting representation of tumor types.The database therefore does not comprehensively cover tumor diversity.
- Acquisition constraints: The system captured tumors only when they were superficial or sufficiently accessible for focusing, constraining the anatomical cases represented.This boundary follows from the demonstrator’s acquisition capability.
- Acquisition constraints: Pushbroom spatial scanning increases acquisition time, while brain motion and surgical artifacts can disrupt image spatial coherence.Snapshot cameras could acquire scenes in a single shot, but provide approximately 10 times fewer spectral bands than pushbroom cameras.
- Data and patient coverage: Limited patient availability prevents comprehensive coverage of heterogeneity across patients and brain-tumor types.The authors characterize the resulting database as preliminary and intended for further investigations.
- Labeling scope: The labeling methodology uses intraoperative MRI, surgeon expertise, and pathological analysis to identify suitable regions and tumor tissue.These sources guide labeling rather than providing a complete gold-reference map for every pixel.
- Future investigations: Further work should correlate tissue biological properties with wavelength-specific spectral responses to improve discrimination, especially in surrounding normal-tissue infiltrate.The proposed correlation is intended to support better tumor identification and delineation.
VIII. CONCLUSIONS
The HELICoiD project released the first public in-vivo hyperspectral human-brain image database for brain-cancer detection. It contains labeled surgical data whose repeatability analysis identifies 450–900 nm as the system’s more efficient operating range and whose subsequent use supports tumor-boundary classification and delineation.
- Contributions: The project provides open access to the first public in-vivo hyperspectral human-brain image database specifically for brain-cancer detection.The database is intended to support further in-vivo brain research using hyperspectral imaging.
- Database and results: 36 images from 22 patients were acquired across two hospitals in the VNIR range of 400–1000 nm.The labeled dataset distinguishes normal tissue, cancer-affected tissue, blood vessels, and background elements.
- Database and results: Repeatability analysis identified 450–900 nm as the spectral range where the acquisition system operates more efficiently.The analysis assessed both spectral and spatial repeatability.
- Applications: The database has supported classification maps that identify and delineate tumor boundaries using traditional machine learning and deep learning approaches.It has also been used as input for accelerating hyperspectral-image processing methods.