PaperlayerView exact revision on arXiv

Source-linked AI summary

Lumbar spine segmentation in MR images: a dataset and a public benchmark

Jasper W. van der Graaf, Miranda L. van Hooff, Constantinus F. M. Buckens, Matthieu Rutten, Job L. C. van Susante, Robert Jan Kroeze, Marinus de Kleuver, Bram van Ginneken, Nikolas Lessmann

arXiv:2306.12217v3eess.IVcs.CV

TL;DR

Existing lumbar spine MRI datasets are limited in size or anatomical coverage, despite the importance of MRI for low back pain assessment. This paper presents a multi-center dataset built through iterative annotation and reports comparable reference performance for its baseline and nnU-Net. It also provides a sequestered test set for ongoing algorithm evaluation.

Abstract

from arXiv · show

This paper presents a large publicly available multi-center lumbar spine magnetic resonance imaging (MRI) dataset with reference segmentations of vertebrae, intervertebral discs (IVDs), and spinal canal. The dataset includes 447 sagittal T1 and T2 MRI series from 218 patients with a history of low back pain and was collected from four different hospitals. An iterative data annotation approach was used by training a segmentation algorithm on a small part of the dataset, enabling semi-automatic segmentation of the remaining images. The algorithm provided an initial segmentation, which was subsequently reviewed, manually corrected, and added to the training data. We provide reference performance values for this baseline algorithm and nnU-Net, which performed comparably. Performance values were computed on a sequestered set of 39 studies with 97 series, which were additionally used to set up a continuous segmentation challenge that allows for a fair comparison of different segmentation algorithms. This study may encourage wider collaboration in the field of spine segmentation and improve the diagnostic value of lumbar spine MRI.

Background & Summary

Lumbar MRI is widely used for low back pain, but existing MRI segmentation datasets are limited in size, anatomical coverage, or dimensionality. This study presents a multi-center dataset, a continuous evaluation challenge, and reference results for two segmentation algorithms.

Background & Summary

  • Automatic image analysis could support more objective and quantitative MRI interpretation, while only 13% of lumbar MRI scans contributed to clinical decision making in one cited US study.Lumbar MRI for low back pain is one of the most-used musculoskeletal imaging procedures.Source: Background & Summary · p. 1
  • Existing lumbar MRI datasets are small, limited to vertebral bodies or midsagittal slices, or cover only vertebrae or IVDs.The paper motivates a larger dataset covering multiple structures relevant to low back pain.Source: Background & Summary · p. 1
  • The study aims to provide a large multi-center dataset with vertebra, IVD, and spinal canal segmentations and per-level radiological gradings.These are the first of the study's three stated goals.Source: Background & Summary · p. 2
  • The study also introduces a continuous challenge for model evaluation and provides reference performance metrics for an IIS baseline and nnU-Net.The two algorithms automatically segment all three spinal structures.Source: Background & Summary · p. 2

Methods

The dataset combines lumbar MRI studies from four Dutch hospitals and uses an iterative annotation workflow to build reference segmentations. It also reserves a patient-level test set for public benchmarking, while using bottom-up vertebral labels that do not establish anatomical identity.

Data collection.

  • MRI studies were collected retrospectively from four Dutch hospitals, with acquisitions spanning January 2019 to March 2022.The contributing centers included a university medical center, two regional hospitals, and an orthopedic hospital.Source: Data collection · p. 2
  • The university center contributed T1, T2, and high-resolution T2 SPACE series, while the other centers required at least a sagittal T1 or T2 series.Four studies were excluded from the university-center data for image quality too low for fully manual segmentation.Source: Data collection · p. 2

Segmentation data.

  • The annotation workflow trained an automatic method on a small annotated subset, then generated initial segmentations for remaining images that were reviewed and manually corrected over successive retraining rounds.The process was repeated until the dataset was annotated; the description specifies four rounds of adding newly annotated data to the training set.Source: Segmentation data · pp. 2–3
  • An initial set of 20 high-resolution T2 SPACE studies was manually annotated, and the corresponding standard T1 and T2 segmentations were resampled and reviewed for misalignment.A medical trainee performed the annotations, which were reviewed by JG.Source: Segmentation data · p. 2
  • Reference labels number vertebrae from the most caudal upward, but those labels are not anatomical identities such as L5.Additional imaging needed to establish anatomical labels was unavailable for most studies, so traditional clinical nomenclature was considered infeasible.Source: Segmentation data · p. 3
  • The 39-study test set was withheld from the public dataset for benchmarking, and studies from the same patient were kept in one split.The challenge permits users to replicate the paper's baseline results.Source: Segmentation data · p. 3

Radiological gradings.

The dataset includes per-level radiological gradings to help users identify healthier and more diseased cases. An experienced musculoskeletal radiologist manually scored seven degenerative features at each IVD level.

Radiological gradings.

  • The gradings are intended to help users distinguish more healthy from more diseased cases in the dataset.Source: Radiological gradings · p. 3
  • The gradings cover Modic changes, endplate changes or Schmorl nodes, spondylolisthesis, disc herniation, disc narrowing, disc bulging, and Pfirrmann grade.Some features are graded by presence or severity; others are binary.Source: Radiological gradings · p. 3
  • An expert musculoskeletal radiologist with 26 years of experience manually scored all features per IVD level.Source: Radiological gradings · p. 3

Baseline 1: Iterative instance segmentation.

The IIS baseline extends an iterative vertebra-segmentation method to segment vertebrae, IVDs, and the spinal canal one level at a time. Separate memory states track segmentation progress, and a 3D U-net-like network processes sagittal MRI patches.

Baseline 1: Iterative instance segmentation.

  • The IIS baseline extends a vertebra-segmentation method to segment one vertebra pair, its inferior IVD, and the spinal canal portion in each 3D patch.The method processes the MR volume iteratively, one vertebral level at a time.Source: Baseline 1: Iterative instance… · p. 3

Instance memory.

  • Separate memory states track vertebra, IVD, and spinal canal progress, but only vertebra and IVD memory are fed to the network as inputs.The spinal canal is elongated, so its memory tracks progress without serving as an extra network input; the network predicts visible canal portions for stitching across patches.Source: Instance memory · pp. 3–4

Network architecture.

  • A single 3D U-net-like network processes 64 × 192 × 192-voxel patches at 2 × 0.6 × 0.6 mm resolution and predicts the three anatomical structures.The patch choice reflects sagittal MRI's higher slice thickness while retaining higher in-plane resolution and fitting a vertebra within a patch.Source: Network architecture · p. 4, Figure 2 · p. 4

Iterative segmentation approach.

The baseline segments the lumbar spine through a moving patch and iterative updates to memory states, alternating vertebrae with IVD and spinal-canal segmentation. It also predicts vertebral completeness, while training uses segmentation, labeling, and completeness losses.

Iterative segmentation approach.

  • The patch follows detected vertebrae through the volume, updating memory states with vertebra, underlying IVD, and spinal-canal masks until no fragments remain or the volume top is reached.After finding the lowest vertebra, the patch recenters on its fragment and continues when the detected vertebra’s volume stabilizes.Source: Iterative segmentation approach · pp. 4–5, Figure 3 · p. 5

Completeness and label prediction.

  • A binary output predicts whether a vertebra is complete; anatomical-label prediction was unused because accurate labels for lumbosacral transitional vertebrae were unavailable.Source: Completeness and label prediction · p. 5

Training of the algorithm.

  • Training combines segmentation error, absolute labeling error, and binary-cross-entropy completeness error, with validation data monitoring training.Images were resampled to 2 × 0.6 × 0.6 mm and augmented with elastic deformation, noise, smoothing, and longitudinal cropping.Source: Training of the algorithm · p. 5

Baseline 2: nnU-Net.

The paper reports nnU-Net reference results against the baseline using a 3D full-resolution, five-fold cross-validated model. Evaluation measures overlap and surface accuracy for each structure and MRI sequence on a sequestered test set.

Baseline 2: nnU-Net.

  • A 3D full-resolution nnU-Net was trained on training and validation data with recommended five-fold cross-validation, then compared overall with the IIS baseline.The framework automatically determined preprocessing, architecture, and other training details.Source: Baseline 2: nnU-Net · p. 5

Evaluation.

  • Evaluation used 3D Dice for volume overlap and average absolute surface distance for surface accuracy, separately by structure and by T1 versus T2 sequence.Predicted structures were matched to references by greatest overlap to limit the influence of labeling differences.Source: Evaluation · p. 5
  • The evaluation used a sequestered test set, while Table 2 summarizes the training/validation distribution and Table 3 summarizes radiological gradings by IVD level and patient.Source: Evaluation · p. 5, Table 2 · p. 6, Table 3 · p. 6

Data Records

The dataset contains 447 MRI series from 218 lumbar MRI studies and includes radiological gradings. Images and corresponding segmentation masks are distributed in MHA format.

Data Records

  • The dataset is available under CC-BY 4.0, with radiological gradings supplied per IVD level and study metadata including scanner and acquisition parameters.Source: Data Records · p. 6
  • The dataset comprises 447 sagittal MRI series from 218 low-back-pain studies, with 3,125 vertebrae, 3,147 IVDs, and 447 spinal-canal segmentations.The series include T1- and T2-weighted acquisitions, including high-resolution SPACE sequences.Source: Data Records · p. 6
  • MRI images and their segmentation masks are stored in separate MHA directories and share names formed from study identifier and sequence type.All series from one study share the same study identifier.Source: Data Records · p. 6

Technical Validation

On the hidden test set, IIS achieved strong segmentation scores across vertebrae, IVDs, and the spinal canal, and its results were nearly identical to nnU-Net. The authors describe the iterative annotation strategy as effective while noting potential bias and limits on the initial manual annotations’ representativeness.

Technical Validation

  • Mean Dice scores were 0.93 (±0.05) for vertebrae, 0.85 (±0.10) for IVDs, and 0.92 (±0.04) for the spinal canal; mean ASD values were 0.49 mm (±0.95 mm), 0.53 mm (±0.46 mm), and 0.39 (mm ± 0.45) mm, respectively.The spinal canal was identified in all scans; one of 656 vertebrae and nine of 688 IVDs were not found, and completeness prediction was correct for 650 of 656 vertebrae (99.1%).Source: Technical Validation · p. 6, Table 4 · p. 7
  • IIS and nnU-Net produced nearly identical results, supporting IIS as an accurate annotation tool and a reasonable benchmark for comparison.IIS performance was also described as comparable to other MR segmentation methods in the literature7,25,26.Source: Technical Validation · pp. 7–8, Table 5 · p. 7
  • The iterative annotation approach incorporated corrections into training data over time, reducing manual effort, but could introduce bias into the final dataset.Fully manual segmentation took about five hours on average, compared with about one hour to correct a predicted segmentation; intricate structures still regularly needed manual correction.Source: Technical Validation · p. 8
  • Initial manual annotation used only high-resolution T2 series from one hospital, which may not represent the entire population.The authors identify this as a limitation of the annotation strategy’s initial data scope.Source: Technical Validation · p. 8
  • The dataset is presented as a public resource for training and evaluating algorithms and comparing results, with baseline results available for comparison.The authors state that the dataset is similar in size to VerSe and provides full segmentation of relevant spinal structures on MR images.Source: Technical Validation · p. 8

Usage Notes

A public segmentation challenge is hosted on the Grand Challenge platform to support fair comparison of algorithms using a sequestered test set.

Usage Notes

  • The challenge evaluates algorithms on a sequestered test set containing 97 MRI series from 39 unique patients.The test set is used for fair comparison between algorithms, including both baseline algorithms.Source: Usage Notes · p. 8
  • The challenge is hosted on the Grand Challenge platform.The passage directs users to the SPIDER challenge site.Source: Usage Notes · p. 8
  • The sequestered evaluation set supports fair comparison between different segmentation algorithms.Both baseline algorithms are included in the intended comparisons.Source: Usage Notes · p. 8

Code availability

The IIS baseline training code and weights, nnU-Net code, and evaluation-metric code are publicly available through their respective repositories.

Code availability

  • The IIS baseline training code and trained weights and biases are publicly available.The passage provides the repository address for these materials.Source: Code availability · p. 8
  • The nnU-Net baseline code is available from the repository identified in the paper.The passage attributes the baseline to Isensee et al.Source: Code availability · p. 8
  • The source code used to compute test-set evaluation metrics is published.The passage provides the evaluation-code repository address.Source: Code availability · p. 8

Additional information

The paper provides correspondence and reprint information and states its publisher’s neutrality regarding jurisdictional claims.

Additional information

  • Correspondence and requests for materials should be addressed to J.W.v.d.G.The paper identifies the corresponding contact by initials.Source: Additional information · p. 9
  • Reprints and permissions information is available from the publisher’s website.The passage names nature.com/reprints.Source: Additional information · p. 9
  • The publisher states it remains neutral regarding jurisdictional claims in published maps and institutional affiliations.This is the publisher’s note accompanying the article.Source: Additional information · p. 9
  • The article is licensed under Creative Commons Attribution 4.0, subject to its stated attribution and third-party-material terms.The license permits sharing and adaptation when its conditions are met.Source: Additional information · p. 9
Loading 2306.12217v3…