PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Geodesic Learning for Segmentation and Anatomical Landmarking

Neslisah Torosdagli, Denise K. Liberton, Payal Verma, Murat Sincan, Janice S. Lee, Ulas Bagci

arXiv:1810.04021v1cs.CVcs.LGstat.ML

TL;DR

The paper addresses automated mandible segmentation and landmark localization in CBCT scans, where anatomical variability and imaging artifacts make analysis difficult. It combines segmentation, geodesic landmark learning, and LSTM-based localization, reporting accurate results on challenging scans and performance comparable to manual landmarking.

  • Problem

    Automated segmentation and clinically relevant landmarking remain difficult for highly variable CMF deformities, while manual or semi-automated annotation is tedious and operator-variable.

    Source: §I · p. 1

  • Method

    A three-step pipeline uses a DenseNet-based segmentation network, U-Net geodesic learning for landmarks, and an LSTM to localize closely spaced landmarks.

    Source: §I · p. 2, A. Step 1: Segmentation Network · p. 4, C. Step 3: Localization of… · p. 7

  • Results

    The geodesic landmark method performed comparably to validated manual landmarking, with median displacement error of 0 mm in most landmarks.

    Source: §V · p. 11

  • Takeaways & Limitations

    The findings support learning-based segmentation and landmarking as a tool for clinical application when anatomical variability is challenging, as in CMF deformity analysis.

    Source: §V · p. 11

    The method operates in pseudo-3D because of memory and hardware limitations, and uses a two-cascaded landmark-localization system.

    Source: §V · p. 11

Abstract

from arXiv · show

In this paper, we propose a novel deep learning framework for anatomy segmentation and automatic landmark- ing. Specifically, we focus on the challenging problem of mandible segmentation from cone-beam computed tomography (CBCT) scans and identification of 9 anatomical landmarks of the mandible on the geodesic space. The overall approach employs three inter-related steps. In step 1, we propose a deep neu- ral network architecture with carefully designed regularization, and network hyper-parameters to perform image segmentation without the need for data augmentation and complex post- processing refinement. In step 2, we formulate the landmark localization problem directly on the geodesic space for sparsely- spaced anatomical landmarks. In step 3, we propose to use a long short-term memory (LSTM) network to identify closely- spaced landmarks, which is rather difficult to obtain using other standard detection networks. The proposed fully automated method showed superior efficacy compared to the state-of-the- art mandible segmentation and landmarking approaches in craniofacial anomalies and diseased states. We used a very challenging CBCT dataset of 50 patients with a high-degree of craniomaxillofacial (CMF) variability that is realistic in clinical practice. Complementary to the quantitative analysis, the qualitative visual inspection was conducted for distinct CBCT scans from 250 patients with high anatomical variability. We have also shown feasibility of the proposed work in an independent dataset from MICCAI Head-Neck Challenge (2015) achieving the state-of-the-art performance. Lastly, we present an in-depth analysis of the proposed deep networks with respect to the choice of hyper-parameters such as pooling and activation functions.

I. INTRODUCTION

The paper targets automated mandible segmentation and landmark localization in CBCT scans, where anatomical variability and image artifacts make analysis difficult. It proposes a three-step deep-learning approach to support more accurate, easier, and faster measurements than manual methods.

I. INTRODUCTION

  • CBCT supports diagnosis and treatment planning but has greater noise and artifact presence than spiral CT, complicating image analysis.Source: §I · p. 1
  • Automatic segmentation and landmarking remain challenging in highly variable CMF deformities, despite their importance for surgical planning and clinical assessment.Manual or semi-automated CT landmarking is tedious and prone to operator variability.Source: §I · p. 1
  • Deformities, missing bones or teeth, implants, and scanner artifacts create anatomical and image variation that existing dental-focused methods do not address broadly.The paper identifies a need for an automated platform for extremely challenging CMF deformities.Source: §I · pp. 1–2
  • The study seeks a fully automated method robust to congenital deformities, developmental variation, age and sex diversity, implants, fillings, and missing anatomy.Source: §I · p. 2
  • The proposed pipeline combines a segmentation CNN, geodesic-map landmark learning, and LSTM-based sequential modeling of landmark relationships.The authors frame the landmarking process as part of segmentation to make geometric measurements more accurate, easier, and faster than manual methods.Source: §I · p. 2

II. RELATED WORK

Prior mandible and CMF methods face difficulties with anatomical variation, memory demands, or dependence on template-based localization. The paper proposes a fully automated geodesic-learning approach with separate handling of sparsely and closely spaced landmarks, alongside analysis of network design choices.

II. RELATED WORK

  • Existing registration-based methods can perform poorly with age differences, missing teeth or anatomy, and imaging artifacts; a seed-localization error can also impair template-based landmarking.Source: §II · p. 2
  • A prior joint CMF segmentation and landmarking method reported 93.27±0.97% dice similarity and mean digitization error below 1.5 mm, but its 3D displacement maps limited landmark count and included redundant slices.Source: §II · p. 3

A. Our Contribution

  • The proposed system injects geodesic maps into deep learning, unlike cited deep approaches that annotate landmarks in Euclidean space,.Source: A. Our Contribution · p. 3
  • A U-Net learns relationships among sparsely spaced landmarks on the bone, while an LSTM treats landmarks as states to identify closely spaced locations in geodesic space.The authors describe this approach as computationally more feasible.Source: A. Our Contribution · p. 3
  • The paper analyzes growth rate in segmentation, pooling functions in detection and segmentation, and the interaction of dropout regularization with pooling.Source: A. Our Contribution · p. 3
  • The proposed geodesic deep-learning method is reported to robustly segment mandibles and provide highly accurate landmarks on an extremely challenging dataset.Source: A. Our Contribution · pp. 3–4

III. METHODS

The method combines a Tiramisu-based mandible segmentation network, geodesic landmark learning, and LSTM classification of closely spaced landmarks. The segmentation network is adapted and evaluated through choices of pooling, regularization, and architecture parameters.

III. METHODS

  • The three-step system segments the mandible, learns landmark relationships on geodesic maps, then uses an LSTM to classify closely spaced landmark locations.The steps use a Tiramisu-based segmentation network, a U-Net-based geodesic learning network, and an LSTM classifier.Source: §III · p. 4

A. Step 1: Segmentation Network

  • Average pooling replaces default pooling because the authors hypothesize it identifies object extent, whereas max pooling identifies only discriminative regions.The proposed rationale is that extent is more useful for pixel-level predictions.Source: A. Step 1: Segmentation Network · p. 5
  • The study tests dropout with batch normalization and pooling, varies dense-block growth rate, and tunes regularization and network depth to avoid CRF post-processing.The paper notes mixed evidence on these regularizers and that optimal growth rate for segmentation had not been explored.Source: A. Step 1: Segmentation Network · p. 5
  • The adapted Tiramisu uses 11 dense blocks and 103 convolutional layers to segment 2D sagittal CBCT slices, trained from scratch without augmentation or complex post-processing.The network has approximately 9M trainable parameters.Source: A. Step 1: Segmentation Network · pp. 4–5

B. Step 2: Geodesic Learning for Landmarking

The method learns landmark relationships on the mandible surface rather than in image-space displacement maps. Signed-distance inputs feed a U-Net that predicts geodesic maps whose extrema identify landmark locations.

B. Step 2: Geodesic Learning for Landmarking

  • Rather than generate three Euclidean displacement maps per landmark, the method uses geodesic distance transforms to learn landmark relationships directly on the mandible surface.The cited displacement-map approach does not incorporate the mandible shape and creates three maps per landmark.Source: B. Step 2: Geodesic Learning for… · pp. 5–6
  • For each landmark, the geodesic map records shortest surface-path distances from that landmark to mandible voxels, computed with Dijkstra’s algorithm.The method describes these surface distances as geodesic distances.Source: B. Step 2: Geodesic Learning for… · p. 6
  • A pixel-wise hard minimum combines multiple landmark maps, and the resulting map’s extrema indicate the landmark locations.The combined map includes location information for all landmarks.Source: B. Step 2: Geodesic Learning for… · p. 6
  • A signed distance transform of the segmented mandible provides the input to an improved Zhang U-Net, which predicts a geodesic map locating the landmarks.The network uses 2D slices of the signed distance transform as input and produces the geodesic map as output.Source: B. Step 2: Geodesic Learning for… · p. 6

C. Step 3: Localization of Closely-Spaced Landmarks

Because fused geodesic maps can be unstable for closely spaced landmarks, the method separates close and sparse landmark localization. It uses an LSTM on sagittal boundary-image rows to identify the close landmarks.

C. Step 3: Localization of Closely-Spaced Landmarks

  • When landmarks are close, fused geodesic-map extrema can be unstable and failed to localize Menton and nearby mid-sagittal landmarks within the stated clinically acceptable error range of ≤3 mm.The authors therefore divide landmark localization into separate close- and sparse-landmark cases.Source: C. Step 3: Localization of… · p. 6
  • The four closely spaced landmarks are Infradentale, B point, Pogonion, and Gnathion; the remaining five are treated as sparsely spaced.The close group is defined as landmarks tied together, while the sparse group occupies five mandible regions.Source: C. Step 3: Localization of… · p. 7
  • The LSTM processes rows of a 64 × 64 sagittal boundary image containing Menton and outputs binary landmark versus non-landmark locations.The input landmarks are ordered along the same sagittal plane; the training set has only 40 such slices, so PCA is used to increase training images.Source: C. Step 3: Localization of… · pp. 7–8, Figure 6 · p. 7

D. Training Framework: End-to-end vs. Sequential vs Mixed

The paper compares sequential training with a mixed approach that trains the first two modules end-to-end and the third sequentially. The third module’s differentiability and convergence requirements prevent fully end-to-end LSTM training.

D. Training Framework: End-to-end vs. Sequential vs Mixed

  • Sequential training fits the three networks individually before engineering their concatenation; mixed training fits the first two end-to-end and adds the third sequentially.The authors call the second approach “mixed.”Source: D. Training Framework: End-to-end vs.… · p. 8
  • The third module’s slice selection is not differentiable, and LSTM localization requires the first two modules to be near convergence, preventing truly end-to-end training.Source: D. Training Framework: End-to-end vs.… · p. 8
  • End-to-end training requires every module to be differentiable with respect to the loss; non-differentiable or sparse modules may yield inferior overall results and.Source: D. Training Framework: End-to-end vs.… · p. 8

A. Data description:

The evaluation uses cross-validation on diverse CBCT scans, multiple expert annotations, and standard segmentation metrics. The proposed network outperformed baseline U-Net, and the paper also reports results on an independent challenge dataset.

A. Data description:

  • The 50-patient CBCT cohort included congenital and developmental anomalies, trauma, and surgical intervention; 5-fold validation used 20,480 training and 5,120 test images per fold.Scans were resampled to 256 × 256 × 512 to reduce computational cost.Source: A. Data description · p. 8

B. Evaluation metrics and annotations:

C. Evaluation of Segmentation

  • The proposed network outperformed baseline U-Net; DSC differed significantly from baseline (P = 0.02), but not manual segmentation (P = 0.77) or improved Zhang’s U-Net (P = 0.28).The proposed Tiramisu network also converged faster than improved Zhang’s U-Net despite having more parameters.Source: C. Evaluation of Segmentation · p. 9, Table V · p. 9
  • On MICCAI Head-Neck Challenge 2015, Dice accuracy was 93.86% versus 90%.Source: C. Evaluation of Segmentation · p. 9, Table VI · p. 9
  • Challenge scans included artifacts and different diseases; connected-component analysis and 3D fill reduced false positives and false negatives.The authors attributed slightly lower mixed-training performance to more parameters and insufficient dataset size, while sequential training was sufficient for segmentation.Source: C. Evaluation of Segmentation · p. 9, Table VI · p. 9

D. Evaluation of Landmark Localization

All nine landmark errors were below three pixels, with most volume-space errors below 1 mm. Localization remained robust across segmentation-accuracy groups, although Menton errors could propagate to closely spaced landmarks.

D. Evaluation of Landmark Localization

  • All nine landmarks had pixel-space errors below 3 pixels; volume-space mean and median errors were below 1 mm for seven landmarks and below 1.36 mm for B point and Pogonion.The authors compare the pixel-space errors with an average 3-pixel inter-observer error and describe the results as clinically acceptable.Source: D. Evaluation of Landmark Localization · p. 10
  • The method detected four landmarks when the left condyle-ramus unit was congenitally absent, but did not locate the fifth landmark on the missing bone.Landmarking also succeeded in examples with chin advancement and bilateral surgical implants.Source: D. Evaluation of Landmark Localization · p. 10
  • With segmentation Dice grouped at ≤90% versus >90%, landmarking errors never exceeded 3 pixels, but a misplaced Menton could propagate slice-spacing-scaled error to closely spaced landmarks.For a Menton offset of n slices with spacing d, the propagated error is n × d mm.Source: D. Evaluation of Landmark Localization · p. 10, Figure 10 · p. 11
  • Max pooling consistently outperformed other pooling methods for landmarking, and all average and median localization errors were below the stated 3 mm clinically acceptable limit.The authors contrast this with segmentation, where average pooling was most effective for pixel-level predictions.Source: D. Evaluation of Landmark Localization · pp. 10–11, Table VII · p. 10

V. DISCUSSION AND CONCLUSION

The proposed networks converged in around 20 epochs, and geodesic-space landmark localization performed comparably to validated manual landmarking. The discussion identifies hardware and cascade constraints while pointing to future cohort analyses and joint modeling.

V. DISCUSSION AND CONCLUSION

  • Geodesic-space landmark localization performed comparably to validated manual landmarking, with median displacement error of 0 mm for most landmarks.The proposed networks also achieved high accuracy on a challenging CBCT dataset and converged in around 20 epochs.Source: §V · p. 11
  • The discussion notes that U-Net-type architectures can incur high parameter counts, and identifies Capsules as a possible future alternative.It describes dense connections, skip connections, and residual blocks as architectural additions used to improve segmentation accuracy in particular applications.Source: §V · p. 11
  • The study handles segmentation and landmarking separately; combining them in a multi-task learning platform is proposed as a natural extension, similar to Zhang et al..The discussion notes that authors in considered both processes in the same platform.Source: §V · p. 11
  • Memory and hardware limits required pseudo-3D processing, while fully 3D work is identified as a possible future extension.The method also uses two cascaded systems for landmark localization; a single network is described as a potential alternative.Source: §V · p. 11
  • Future studies are proposed to apply the algorithms to large cohorts and establish normative craniofacial datasets.The authors also identify rare craniofacial disorders with anatomical variation as a potential beneficiary of accurate landmark localization.Source: §V · p. 11
  • Learning-based segmentation and landmarking are presented as promising for clinical analysis of challenging anatomical variability, including CMF deformity analysis.The study reports highly accurate and efficient results from 50 extremely challenging CBCT scans.Source: §V · p. 11

APPENDIX

The appendix compares pooling and regularization choices for segmentation. Average pooling was the most robust pooling option reported, including when batch normalization and dropout were used together.

1) Effect of pooling functions:

2) Disharmony between BN and dropout:

  • With batch normalization, dropout was often detrimental to segmentation except at a 20% drop rate.Table VIII compares segmentation accuracies across regularization choices; a drop ratio of 0 denotes no dropout.Source: 2) Disharmony between BN and dropout · p. 12, Table VIII · p. 12
  • When batch normalization and dropout were used together, average pooling was the most robust pooling function among those tested.The comparison concerns the interaction of batch normalization, dropout, and pooling choices in segmentation.Source: 2) Disharmony between BN and dropout · p. 12

3) The role of growth rate in dense blocks:

For segmentation, the authors report that Tiramisu performed best with growth rate 24 and dropout 0.2, while ReLU slightly outperformed Swish. Qualitative review by two experts covered 250 scans, with low scores typically associated with serious anatomical deformations.

3) The role of growth rate in dense blocks:

  • Tiramisu with growth rate 24 and drop rate 0.2 produced the best segmentation accuracies in the experiments.Without dropout, growth-rate performance reversed, which the authors interpret as indicating dropout's regularizing impact.Source: 3) The role of growth rate in dense… · p. 12

4) The choice of activation functions:

  • ReLU and Swish showed no significant performance difference, although ReLU yielded slightly better results in all sub-experiments.Swish was tested by replacing ReLU in the 103-layer Tiramisu network.Source: 4) The choice of activation functions · p. 12

B. Qualitative Evaluation

  • About 5% of the 250 segmentation results received score 1 from both experts, and scores of 1 or 2 typically occurred with serious anatomical deformations.Scores 3 and 4 represent clinically acceptable segmentations; Figure 12 shows examples across scores 1–4.Source: B. Qualitative Evaluation · p. 12, Figure 11 · p. 12, Figure 12 · p. 13
  • Two experts visually scored the segmentations of 250 patient scans, with scores from 0 to 4 and score 0 reserved for scans where automatic mandible segmentation was not possible.The figure captions identify expert evaluation of the scans and present examples of score levels.Source: B. Qualitative Evaluation · p. 12, Figure 11 · p. 12, Figure 12 · p. 13

Related papers

Loading 1810.04021v1…