PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Muscle Segmentation from Clinical CT using Bayesian U-Net for Personalized Musculoskeletal Modeling

Yuta Hiasa, Yoshito Otake, Masaki Takao, Takeshi Ogawa, Nobuhiko Sugano, Yoshinobu Sato

arXiv:1907.08915v2eess.IVcs.CV

TL;DR

Individual-muscle segmentation from clinical CT remains time-consuming and expert-dependent. The paper applies Bayesian U-Net with Monte Carlo dropout to segment muscles and estimate uncertainty. On 20 fully annotated CTs, it reports improved segmentation metrics over state-of-the-art methods and demonstrates uncertainty-based accuracy prediction and annotation-cost reduction.

  • Problem

    Manual segmentation of individual-muscle geometry from subject-specific medical images is time-consuming and requires expert knowledge, limiting clinical use.

    Source: §I · p. 1

  • Method

    Bayesian U-Net uses dropout sampling to segment individual muscles and estimate uncertainty, which the authors apply to accuracy prediction and active-learning annotation selection.

    Source: D. Bayesian active learning · p. 4

  • Results

    The 20-CT LOOCV achieved DC 0.891±0.016 and ASD 0.994±0.230 mm, significant improvements over state-of-the-art methods.

    Source: §IV · p. 7

  • Takeaways & Limitations

    The method can support patient-specific muscle analysis and active learning, with the latter reducing manual annotation cost by approximately 90-fold.

    Source: §IV · pp. 9–10

    The training and test data have limited variation; the active-learning evaluation was a simulation, and the TCIA ground truth covers only gluteus medius.

    Source: §IV · p. 10

Abstract

from arXiv · show

We propose a method for automatic segmentation of individual muscles from a clinical CT. The method uses Bayesian convolutional neural networks with the U-Net architecture, using Monte Carlo dropout that infers an uncertainty metric in addition to the segmentation label. We evaluated the performance of the proposed method using two data sets: 20 fully annotated CTs of the hip and thigh regions and 18 partially annotated CTs that are publicly available from The Cancer Imaging Archive (TCIA) database. The experiments showed a Dice coefficient (DC) of 0.891 +/- 0.016 (mean +/- std) and an average symmetric surface distance (ASD) of 0.994 +/- 0.230 mm over 19 muscles in the set of 20 CTs. These results were statistically significant improvements compared to the state-of-the-art hierarchical multi-atlas method which resulted in 0.845 +/- 0.031 DC and 1.556 +/- 0.444 mm ASD. We evaluated validity of the uncertainty metric in the multi-class organ segmentation problem and demonstrated a correlation between the pixels with high uncertainty and the segmentation failure. One application of the uncertainty metric in active-learning is demonstrated, and the proposed query pixel selection method considerably reduced the manual annotation cost for expanding the training data set. The proposed method allows an accurate patient-specific analysis of individual muscle shapes in a clinical routine. This would open up various applications including personalization of biomechanical simulation and quantitative evaluation of muscle atrophy.

I. INTRODUCTION

Subject-specific muscle geometry can improve biomechanical simulations, but manually segmenting individual muscles is time-consuming and requires expertise. Prior work leaves clinical CT segmentation especially challenging, motivating an automated approach.

I. INTRODUCTION

  • Volumetric subject-specific muscle models can improve simulation accuracy, but expert-intensive manual segmentation of individual muscles precludes routine clinical use.Prior work often represented muscles as lines between origin and insertion; volumetric representations capture subject-specific geometry.Source: §I · p. 1

A. Related work

  • For CT, lower soft-tissue contrast than MRI makes individual-muscle segmentation more difficult, especially in densely packed hip and thigh regions.Source: A. Related work · p. 1
  • The hierarchical multi-atlas method segmented 19 muscles from origin to insertion, reported average Dice 0.838, and required 41 minutes per CT on a 60-core server.Its hierarchy progressed from skin surface to all muscle tissue and then individual muscles.Source: A. Related work · p. 2
  • The authors propose CNN-based CT segmentation to improve accuracy and speed, while using Bayesian deep learning uncertainty for accuracy prediction and active-learning annotation reduction.The cited uncertainty approach estimates model uncertainty from differences between test samples and training data, and the two applications connect to and,.Source: A. Related work · p. 2

B. Contributions

The paper evaluates CNN-based segmentation of 19 individual hip and thigh muscles and examines uncertainty for predicting segmentation accuracy and selecting annotations in active learning.

B. Contributions

  • The study reports significantly improved CNN segmentation accuracy for 19 individual muscles in CTs of the hip and thigh regions.Source: B. Contributions · p. 2
  • It evaluates Bayesian U-Net on 20 fully annotated clinical CTs and 18 partially annotated TCIA CTs, including uncertainty-based accuracy prediction and active-learning sample selection.Source: B. Contributions · p. 2

C. Paper organization

  • The paper describes methods in Section II, evaluates them on two data sets in Section III, and discusses results and conclusions in Section IV.Source: C. Paper organization · p. 2

A. Overview

The method segments skin before individual muscles, predicts uncertainty and estimates each muscle's Dice coefficient; its evaluation uses fully annotated clinical CTs and a separate partially annotated TCIA set.

A. Overview

  • Skin is segmented with a 2D U-Net to isolate the body; Bayesian U-Net then segments muscles and predicts uncertainty, which is regressed to estimate Dice without ground truth.The active-learning framework was tested in a simulated setting with initially partial annotations and iterative slice annotation.Source: A. Overview · p. 3

B. Data sets

  • The evaluation uses a fully annotated, non-public clinical CT data set and a partially annotated, publicly available CT data set.Source: B. Data sets · p. 3

1) Osaka University Hospital THA data set (THA data set):

  • The THA set contains 20 CT volumes with 19 muscles and three bones manually annotated; annotation took about 40 hours per volume.The volumes were resampled to 1.0 mm slice intervals; Fig. 2 visualizes muscles by functional group and region, with anterior and posterior views.Source: 1) Osaka University Hospital THA data… · p. 3, Figure 2 · p. 3

2) TCIA soft tissue sarcoma data set (TCIA data set):

  • The TCIA set comprises 18 hip-region CT volumes, with gluteus medius labels verified by an expert surgeon; it was reserved for evaluating generalizability.It was not used for training or parameter tuning.Source: 2) TCIA soft tissue sarcoma data set… · p. 3

C. Estimation of uncertainty metric

The method approximates Bayesian prediction with dropout sampling and uses predictive variance for uncertainty. It introduces structure-wise metrics and estimates their Dice relationship through cross-validation.

C. Estimation of uncertainty metric

  • Dropout-masked weights approximate the posterior distribution, and the predictive distribution is estimated by Monte Carlo dropout sampling.The output mean and variance across samples serve as the segmentation result and uncertainty estimate, respectively.Source: C. Estimation of uncertainty metric · pp. 3–4
  • The proposed predictive structure-wise variance (PSV) and predictive Dice coefficient (PDC) are structure-wise uncertainty metrics for multi-class segmentation.PSV represents predictive variance per unit area of pixels classified as the target structure.Source: C. Estimation of uncertainty metric · p. 4
  • The authors estimate linear coefficients and bias for the uncertainty metric with K-fold cross-validation using Dice and PSV observations.K-1 groups train each model while the remaining group supplies the evaluation observations.Source: C. Estimation of uncertainty metric · p. 4
  • Bayesian U-Net inserts dropout before each max-pooling layer and after each up-convolution layer, extending U-Net with MC dropout.The paper calls this architecture Bayesian U-Net.Source: C. Estimation of uncertainty metric · p. 4

D. Bayesian active learning

The active-learning procedure selects uncertain slices and then uncertain pixels for expert annotation. It combines manual labels for uncertain pixels with predicted labels for certain pixels, with a threshold that trades annotation cost against accuracy.

D. Bayesian active learning

  • Active learning interactively expands a small labeled data set using expert input when a larger unlabeled data set is available.The proposed pixel-selection method aims to reduce the number of pixels queried to experts.Source: D. Bayesian active learning · p. 4
  • The method selects uncertain slices from unlabeled data, then chooses representative slices through similarity-based clustering before selecting uncertain pixels.The slice-selection step follows Yang et al..Source: D. Bayesian active learning · p. 4
  • Experts label pixels whose uncertainty exceeds threshold T, while predicted labels are used for the remaining pixels.The threshold determines the trade-off between manual annotation cost and achieved accuracy.Source: D. Bayesian active learning · p. 4

E. Implementation details

Implementation includes CT intensity normalization, augmentation, and connected-component post-processing. The comparison evaluates segmentation methods using volume-level error metrics and patient-level plots.

E. Implementation details

  • CT intensities are mapped from [−150, 350] HU to [0, 255], with augmentation by translation, rotation, scaling, shearing, and right-left flipping.The largest connected component is extracted for each muscle in post-processing.Source: E. Implementation details · p. 4

F. Comparison with conventional methods

  • The comparison evaluates the hierarchical multi-atlas method, FCN-8s, and U-Net using Dice coefficient and average symmetric surface distance calculated per volume.Statistical significance was tested with paired t-tests and Bonferroni correction.Source: F. Comparison with conventional methods · p. 4
  • Figure 3 plots patient-level averages across 19 muscles for Dice coefficient and average symmetric surface distance, with 20 data points per method.The plots show quartiles, medians, and outliers.Source: Figure 3 · p. 5

III. RESULTS

The results compare U-Net with conventional methods on annotated clinical CTs and report segmentation accuracy across muscles and bones. They also describe uncertainty validity and implementation-related findings.

A. Network architecture selection and comparison with conventional methods

  • 0.891±0.016 DC and 0.994±0.230 mm ASD: U-Net significantly outperformed the hierarchical multi-atlas method and FCN-8s on 20 clinical CTs (p < 0.01).The evaluation used leave-one-out cross-validation on the 20 labeled CTs, with 19 muscles among 23 classes.Source: A. Network architecture selection and… · p. 5
  • U-Net improved ASD for almost all individual muscles and patients, with five exceptions across the psoas major, gracilis, and semimembranosus.Figure 4’s heatmap compares the hierarchical multi-atlas method [40] and U-Net; blue indicates lower ASD.Source: A. Network architecture selection and… · p. 5, Figure 4 · p. 5
  • U-Net produced more accurate muscle-boundary segmentations than the other two methods in the representative Patient #01 visualization.The bottom slice visualization corresponds to the dotted-line region in the left-most column.Source: A. Network architecture selection and… · p. 5, Figure 5 · p. 6
  • U-Net’s full-resolution fusion supports detail delineation through pixel-wise correspondence, while FCN-8s fuses latent vectors at one-eighth input resolution.The paper attributes FCN-8s’s consistently lower accuracy to lack of details.Source: A. Network architecture selection and… · p. 5
  • For U-Net, DC was 0.981 ± 0.0043 for the pelvis, 0.985 ± 0.0065 for the femur, and 0.962 ± 0.0166 for the sacrum.ASD was 0.145 ± 0.040 mm, 0.175 ± 0.084 mm, and 0.402 ± 0.243 mm, respectively.Source: A. Network architecture selection and… · p. 5
  • Skin-surface segmentation reduced undesirable variation for muscle segmentation at low annotation cost, especially in scans with a calibration phantom near the skin.It did not significantly change accuracy in the THA data set (p > 0.05).Source: A. Network architecture selection and… · p. 5
  • Inference took approximately 2 minutes per CT volume of about 500 slices, and post-processing took about 3 minutes.Average training time was approximately 11 hours for FCN-8s and U-Net on the stated hardware.Source: A. Network architecture selection and… · p. 5
  • The relationship between accuracy and uncertainty supported using Bayesian U-Net’s uncertainty metric as an indicator of unobservable segmentation error without ground truth.Source: A. Network architecture selection and… · p. 6

B. Estimation of uncertainty metric

The study relates estimated uncertainty to segmentation error using 20 labeled CTs. Greater uncertainty was associated with lower Dice coefficient, with one muscle showing relatively low correlation.

B. Estimation of uncertainty metric

  • The overall correlation ratio between uncertainty and Dice coefficient was −0.784, with increased uncertainty corresponding to decreased DC across the evaluated muscles.The obturator internus had relatively low correlation.Source: B. Estimation of uncertainty metric · p. 6

2) Generalization capability to an unseen data set:

The authors report that the uncertainty metric generalizes from the THA data set to TCIA, where higher uncertainty regions coincided with partial segmentation failure.

2) Generalization capability to an unseen data set:

  • 0.011±0.0084 was the mean absolute error between DC and PDC on TCIA, with the prediction calibration fit using 20 THA cross-validation data points.Bayesian U-Net was retrained using all 20 annotated THA CTs.Source: 2) Generalization capability to an… · p. 6
  • For gluteus medius in 18 TCIA patients, average DC was 0.914±0.026 and average ASD was 2.927±4.997 mm.Excluding four patients with extremely large sarcomas, the averages were 0.925±0.014 DC and 1.135±0.777 mm ASD.Source: 2) Generalization capability to an… · p. 6
  • Higher-uncertainty regions coincided with partial segmentation failure, and the uncertainty distribution was similar to that in the THA data set.The authors suggest these results indicate generalization of the uncertainty metric between data sets.Source: 2) Generalization capability to an… · p. 6

C. Bayesian active learning

The simulated active-learning experiment compared pixel-selection strategies as training data accumulated, using uncertainty to guide which pixels experts labeled.

C. Bayesian active learning

  • The simulation trained on 15 CTs and tested on five, adding 5% of each training CT's remaining unlabeled slices per step over 20 steps.Slices were downsampled to 256 × 256, and data augmentation was omitted.Source: C. Bayesian active learning · p. 6
  • The comparison included fully manual, random, and proposed selection of uncertain pixels; random and proposed methods used equal numbers of annotated pixels.MAC quantifies queried pixels relative to pixels in the added label image; fully manual selection had MAC = 1.0.Source: C. Bayesian active learning · pp. 6–7
  • With T <= 2.5 × 10−3, proposed selection achieved DC about 0.03 higher than random selection and close to fully manual selection.Random selection plateaued around DC 0.843, while the other methods continued increasing in accuracy.Source: C. Bayesian active learning · p. 7
  • At T = 2.5 × 10−3, the proposed method reduced annotation cost approximately 90-fold versus fully manual selection, with median MAC 0.0108 over 19 acquisition steps.The authors identify this threshold as a trade-off between achievable accuracy and annotation cost.Source: C. Bayesian active learning · p. 7

IV. DISCUSSION AND CONCLUSION

Bayesian U-Net improved lower-extremity muscle segmentation, estimated segmentation accuracy without ground truth, and showed potential to reduce annotation cost. The discussion also identifies data variation and simulated active learning as limits on the evidence.

IV. DISCUSSION AND CONCLUSION

  • DC of 0.891±0.016 and ASD of 0.994±0.230 mm significantly improved over state-of-the-art methods across 19 muscles in 20 fully annotated CTs.The reported improvements were significant at p < 0.01.Source: §IV · p. 7
  • Higher-uncertainty regions generally agreed with segmentation errors, and TCIA examples included failures associated with tumors.The discussion also notes failures for some muscles that were correctly indicated by their uncertainty metrics.Source: Figure 7 · p. 8, Figure 8 · p. 8, §IV · p. 9
  • High uncertainty correlated with lower Dice scores, supporting its use to predict segmentation accuracy without ground truth, although the obturator internus had low correlation (r = 0.08).The authors attribute the exception possibly to variability in manual tracing becoming dominant.Source: §IV · pp. 7–9
  • The active-learning pixel-selection method reduced annotation cost by approximately 90-fold, while its uncertainty threshold traded off annotation cost, learning speed, and final accuracy.The experiment was a simulation, and choosing a threshold for a new dataset requires further study.Source: §IV · pp. 9–10
  • Automated segmentation may support patient-specific biomechanical modeling and muscle atrophy analyses while reducing manual labor and interoperator variability.The authors also describe CT as faster than MRI for imaging the lower extremity, with their protocol taking less than 30 seconds.Source: §IV · p. 10
  • The main training and test data had limited variation: THA scans included only women undergoing arthroplasty, while TCIA ground truth covered only the gluteus medius.The authors recommend evaluating active learning with a larger labeled and unlabeled CT database in a more realistic clinical scenario.Source: §IV · p. 10

APPENDIX

The appendix outlines similarity-based slice selection and supplements the main evaluation with examples of TCIA failures, skin-surface segmentation, and muscle-structure results.

APPENDIX

  • Similarity-based slice selection iteratively adds representative slices, choosing candidates by their similarity to the unlabeled data set.The appendix labels this procedure as slice selection by similarity-based clustering.Source: APPENDIX · p. 10

SUPPLEMENTARY MATERIALS

  • In TCIA examples, tumor-related mis-segmentations and CT artifacts coincided with high uncertainty; out-of-field muscles also mis-segmented with high uncertainty.The examples include failures in multiple patients and an artifact-related failure in Patient #03.Source: SUPPLEMENTARY MATERIALS · p. 12
  • In three independent QCT cases, skin-surface segmentation corrected phantom mis-segmentation and improved muscle segmentation near the phantom boundary.Without that step, the calibration phantom was wrongly segmented and nearby muscle errors were observed.Source: SUPPLEMENTARY MATERIALS · p. 13
  • Supplementary results compare per-structure accuracy for the hierarchical multi-atlas method, FCN-8s, and U-Net, and list structure volumes and CT-slice counts by patient.The accuracy visualization uses blue to indicate high average accuracy or low variance; the volume visualization uses blue for small relative volume.Source: SUPPLEMENTARY MATERIALS · p. 14

Related papers

Loading 1907.08915v2…