Source-linked AI summary
Iterative fully convolutional neural networks for automatic vertebra segmentation and identification
Nikolas Lessmann, Bram van Ginneken, Pim A. de Jong, Ivana Išgum
TL;DR
Partial spine scans create a need for vertebra segmentation and anatomical identification that do not depend on particular vertebrae or a fixed number of visible instances. The paper uses an iterative fully convolutional network with memory to segment and label vertebrae sequentially, and evaluates it across diverse datasets. It performs favorably against state-of-the-art methods while remaining fast, flexible, and generalizable across imaging settings.
Problem
Automatic spine analysis requires vertebra segmentation and identification, but scans often cover only part of the spine, so methods should not assume particular visible vertebrae or a fixed count.
Method
An iterative fully convolutional network uses instance memory to segment vertebrae one after another while concurrently identifying them and classifying their completeness.
Results
The method outperformed all participants in the CSI 2014 spine segmentation challenge and performed better or comparably to state-of-the-art methods across diverse datasets.
Takeaways & Limitations
The approach is fast, flexible, and accurate across dedicated and non-dedicated scans with varied modalities, fields of view, resolution, dose, and vertebral deformity.
Takeaways & Limitations
Sequential inference is susceptible to cascading failures, especially with extreme anatomical abnormalities or severe imaging artifacts.
Abstract
from arXiv · showhide
Precise segmentation and anatomical identification of the vertebrae provides the basis for automatic analysis of the spine, such as detection of vertebral compression fractures or other abnormalities. Most dedicated spine CT and MR scans as well as scans of the chest, abdomen or neck cover only part of the spine. Segmentation and identification should therefore not rely on the visibility of certain vertebrae or a certain number of vertebrae. We propose an iterative instance segmentation approach that uses a fully convolutional neural network to segment and label vertebrae one after the other, independently of the number of visible vertebrae. This instance-by-instance segmentation is enabled by combining the network with a memory component that retains information about already segmented vertebrae. The network iteratively analyzes image patches, using information from both image and memory to search for the next vertebra. To efficiently traverse the image, we include the prior knowledge that the vertebrae are always located next to each other, which is used to follow the vertebral column. This method was evaluated with five diverse datasets, including multiple modalities (CT and MR), various fields of view and coverages of different sections of the spine, and a particularly challenging set of low-dose chest CT scans. The proposed iterative segmentation method compares favorably with state-of-the-art methods and is fast, flexible and generalizable.
1. Introduction
Automatic vertebra segmentation and identification support downstream spine analysis but must handle partial, variable scan coverage without assuming a fixed number of visible vertebrae. The paper proposes an iterative, supervised approach that segments, labels, and classifies vertebrae sequentially across diverse imaging settings.
- Vertebra segmentation and identification support automated fracture detection, deformity assessment, and computer-assisted spinal surgery.
- A generic algorithm must accommodate different image resolutions and partial coverage of the spine, without assuming how many vertebrae are visible.
- Variable fields of view make anatomical labeling difficult because ribs, sacrum, and neighboring vertebrae provide incomplete or ambiguous identification cues.
- Partially visible vertebrae should be ignored or explicitly marked incomplete before downstream analyses such as vertebral-height measurements.
- The proposed fully convolutional network iteratively performs vertebra detection, segmentation, anatomical identification, and completeness classification end-to-end.
2. Related work
Prior vertebra methods span statistical models, machine learning, deep learning, and generic instance-segmentation frameworks. Much of the earlier machine-learning literature used learning mainly for detection or initialization, while many approaches imposed constraints on visible instances or other features.
- Earlier vertebra segmentation methods predominantly used statistical shape models, atlases, level sets, or other explicit model-fitting approaches.
- Several machine-learning methods detected vertebral centers or bounding boxes before applying separate segmentation procedures.
- Deep-learning approaches increasingly replaced explicit vertebral shape and appearance models with convolutional and recurrent neural networks.
- Anatomical identification has used appearance and shape models, handcrafted-feature machine learning, and deep neural networks.
- Generic convolutional instance-segmentation frameworks commonly separate detection from segmentation and often constrain the number of instances or other features.
3. Methods
The method uses a single patch-based 3D fully convolutional network with instance memory to segment, identify, and assess vertebrae iteratively. Spatial traversal follows adjacent vertebrae, while auxiliary network components handle anatomical labeling and completeness classification.
- 3. Methods: A single fully convolutional network performs vertebra segmentation and identification concurrently, avoiding successive detection, segmentation, and instance-separation stages.The approach analyzes patches rather than entire 3D volumes and supports multiple network tasks within one supervised, end-to-end framework.
- 3.2. Instance memory: The network combines binary voxel segmentation with an instance memory that marks previously labeled voxels and directs attention to the next vertebra.Image and corresponding memory patches form a two-channel input; after each vertebra is segmented, the memory is updated for the next iteration.
- 3.1. Segmentation network: The segmentation network uses a 128 × 128 × 128 voxel patch, isotropically resampling inputs to 1 mm resolution before restoring masks to the original resolution.Its U-net-inspired compression and expansion paths use skip connections, and the architecture also supports the identification and completeness outputs.
- 3.3. Iterative instance segmentation: Traversal begins with sliding-window search, then repeatedly centers the patch on detected vertebral fragments until the current vertebra is fully visible and the position converges.The patch size is selected to contain part of the following vertebra, allowing adjacency knowledge to guide movement along the column.
- 3.3. Iterative instance segmentation: Once a vertebra converges, its uniquely labeled mask is stored and the updated memory causes the next analysis of the same patch to target the following vertebra.This produces a successive top-down or bottom-up traversal rather than random instance order.
- 3.4–3.5. Identification and completeness: The network includes recognition branches for anatomical identification and completeness classification, with incomplete vertebrae retained in memory but excluded from the segmentation mask.The completeness branch predicts a probability of full visibility, while the identification branch predicts the anatomical label.
4. Evaluation
The method was evaluated on five diverse CT and MR datasets spanning dedicated spine imaging, variable spine coverage, compression fractures, and low-dose chest CT. Performance was assessed with overlap, surface-distance, anatomical-identification, and completeness-classification metrics.
- Datasets: Five CT and MR datasets were used, covering dedicated spine scans, incidental chest imaging, lumbar scans, thoracolumbar scans, and T2-weighted lumbar MR.The datasets varied in field of view, spine coverage, subject populations, and imaging modality.
- Datasets: The thoracolumbar CT dataset contained 15 scans visualizing all thoracic and lumbar vertebrae from young adults without vertebral fractures.Scans used intravenous contrast and had 0.31–0.36 mm in-plane resolution.
- Datasets: The xVertSeg.v1 dataset contained 15 lumbar CT scans from subjects with compression fractures of various grades and types.Manual reference segmentations were defined through consensus reading by two observers.
- Datasets: The low-dose chest CT dataset contained 55 lung-screening scans with variable coverage of thoracic and upper lumbar vertebrae.Subjects were heavy smokers aged 50–74 years and therefore had increased risk for vertebral compression fractures due to age and smoking history.
- Evaluation metrics: Segmentation was evaluated with Dice overlap and ASSD surface distance, while identification used accuracy and linearly weighted kappa.Completeness classification was evaluated with classification accuracy and false-positive and false-negative counts per scan.
5. Experiments and Results
Experiments across CT and MR datasets evaluated segmentation, anatomical identification, completeness classification, loss functions, multitask paths, traversal direction, and a multiclass FCN baseline. The iterative approach achieved high segmentation and identification performance across varied scans, while performance depended on image difficulty and design choices.
- Segmentation performance: 94.9 ± 2.1 % average Dice was achieved for CT vertebra segmentation, compared with 94.4 ± 3.3 % for MR vertebral body segmentation.Surface distances were 0.2 ± 0.1 mm on CT and 0.4 ± 0.3 mm on MR.
- Segmentation performance: Segmentation was more accurate on high-resolution dedicated spine CT than on low-dose chest CT and scans containing severe compression fractures.Lumbar vertebrae were more accurately segmented than thoracic vertebrae, which were more often covered by challenging low-dose chest CT scans.
- Comparison with previous methods: The iterative method outperformed previous methods on thoracolumbar and lumbar spine CT datasets, while results on xVertSeg.v1 were comparable to one method and below another under non-directly comparable splits.For MR vertebral body segmentation, it achieved higher average Dice scores and lower surface distances than previous methods, with higher variance than Korez et al. (2016).
- Loss-function comparison: 89.9 ± 6.8 % Dice and 0.5 ± 0.7 mm surface distance were obtained with Dice-score optimization, versus 93.1 ± 1.5 % and 0.3 ± 0.1 mm with the proposed loss on low-dose chest CT.The categorical cross-entropy network converged to predicting every voxel as background.
- Network design and traversal: Adding anatomical labeling and completeness classification paths produced overall comparable segmentation performance to the segmentation-only network.Downward traversal also had comparable overall performance but deteriorated on low-dose chest CT; initializing it from upward traversal was not beneficial.
- Identification performance: 93 % (206/222, κ = 0.99) of CT vertebrae received correct anatomical labels, MR identification succeeded in all cases, and labeling was correct in 97 % of scans.One thoracolumbar CT scan was offset by one vertebra, producing incorrect labels for all 16 visible vertebrae.
- Completeness classification performance: 97 % of CT vertebrae were correctly classified as completely or incompletely visible, and all MR vertebrae were correctly classified.Most errors occurred in low-dose chest CT at the first or last visible vertebra near the field-of-view boundary.
- Comparison with multiclass FCN: The multiclass FCN achieved 78.7 ± 13.5 % average Dice, 8.2 ± 9.4 mm average ASSD, and 89 % anatomical identification, while disregarded class labels yielded 84.6 ± 6.9 % Dice.The individual vertebral instances were often not well separated.
6. Discussion
The iterative FCN approach performs robustly across diverse scans and modalities, while its traversal direction, computational demands, and failure modes define important practical boundaries. The discussion also highlights broad applicability alongside limitations in unusual anatomy and unseen data types.
- Performance and capabilities: The approach outperformed CSI 2014 challenge methods and matched or exceeded state-of-the-art performance, reaching near-interobserver performance on low-dose chest CT.It jointly performs instance segmentation, identification, and completeness classification using memory-informed iterative processing.
- Generalization: Across diverse datasets, the method handled arbitrary fields of view, low-resolution and low-dose scans, deformed vertebrae, and an unseen lumbar CT dataset.It also transferred without modification to vertebral body segmentation in T2-weighted MR images.
- Comparison with multiclass segmentation: The iterative method substantially outperformed a regular multiclass FCN, indicating better separation of individual vertebral instances when the network focuses on one instance at a time.The multiclass FCN remained fairly accurate overall, but its individual vertebrae were not well separated.
- Architecture and resources: The combined multi-task network simplifies training and inference by producing multiple predictions through one network, but its extra output paths increase GPU-memory demands.Larger-memory hardware could permit larger networks and potentially improve performance.
- Traversal strategy: Upwards traversal was more robust than downwards traversal on low-dose chest CT, mainly because the smallest and artifact-affected upper vertebrae are encountered first when traversing downwards.The other CT datasets showed no substantial traversal-direction difference.
- Evaluation scope: The evaluation omitted cervical scans and patients with some important data characteristics, limiting evidence for those settings.The authors note that the diverse datasets nevertheless covered multiple modalities, fields of view, and spine sections.
- Limitations: Anatomical labeling can fail for patients with irregular numbers of vertebrae because such cases were absent from training data.The authors suggest that a global labeling model, such as a Markov model, could address related labeling weaknesses.
- Limitations: The method is susceptible to cascading failures: missing or missegmenting one vertebra can affect all subsequent vertebrae, especially with extreme abnormalities or severe artifacts.The approach also lacks an explicit mechanism ensuring that a predicted mask covers only one vertebra when multiple vertebrae are visible.