PaperlayerView exact revision on arXiv

Source-linked AI summary

A Joint 2D-3D Statistical Shape Model for Orthopedic Reconstruction

Florence Dell'Aniello Picard, Pranav Poudel, Nairouz Shehata, Frédéric Lavoie, Herve Lombaert

arXiv:2609.09010v1cs.CV

TL;DR

Radiograph-based femoral reconstruction is ill-posed because projections lose depth, while existing SSM approaches remain computationally costly and single-dimensional. This paper learns joint 2D–3D shape variation and directly recovers 3D shapes from 2D observations, outperforming a 3D SSM baseline with roughly four-times-faster inference.

  • Problem

    Radiographs discard depth, making 3D reconstruction ill-posed, and existing SSM approaches are computationally costly and limited to a single geometric dimensionality.

    Source: §1 · p. 2

  • Method

    The method jointly models paired 2D and 3D stationary velocity fields in a shared latent space, then lifts inferred 2D latent coordinates into the 3D subspace.

    Source: §1 · p. 3

  • Results

    DSC improves from 94.96% to 96.15% for the joint 2D–3D SSM versus the 3D SSM baseline, with inference in 2.76 s, roughly four times faster.

    Source: §4 · p. 10

  • Takeaways & Limitations

    The approach reconstructs 3D femoral shapes from 2D segmentations without iterative optimization, with qualitative improvements at the condyles, important for surgical planning.

    Source: §4 · p. 10

    The method assumes known, fixed calibration, which limits its applicability when calibration data are unavailable or patient positioning varies significantly.

    Source: §4 · p. 10

Abstract

from arXiv · show

Three-dimensional femoral reconstruction from radiographs supports surgical planning, implant sizing, and post-operative follow-up, but remains ill-posed as X-ray projections discard depth information. Existing methods often incorporate a 3D statistical shape model (SSM) as a shape prior to guide reconstructions toward anatomically plausible shapes, relying on iterative 3D-to-2D projection matching. Yet, these approaches are computationally expensive and constrain their SSM to a single dimensionality, leaving the statistical relationship between 2D observations and 3D geometry largely unexploited and unexplored. We instead propose a joint 2D-3D SSM that explicitly captures the co-variation between 2D and 3D segmentations in a shared latent space. During training, 2D and 3D segmentations are registered to a common 3D template and its corresponding 2D projections, and the resulting stationary velocity fields are jointly decomposed using principal component analysis (PCA). This joint modeling allows the 2D-to-3D mapping to be learned directly from data rather than computing correspondences at inference time. For unseen subjects, the 3D shape is recovered directly by lifting the 2D latent coordinates to the 3D PCA subspace, thereby eliminating the need for iterative 3D-to-2D projection. Experiments on NMDID demonstrate that the proposed joint 2D-3D SSM outperforms a widely-used 3D-only SSM baseline while achieving inference approximately 4 times faster, at under 3 seconds per subject. The code is available at: https://github.com/florence-dellaniello-picard/joint2d3d-ssm.

1 Introduction

Femoral reconstruction from radiographs is clinically useful but ill-posed because projections lose depth, while existing SSM approaches are computationally costly and model only one dimensionality. The proposed joint model learns 2D–3D shape co-variation and reconstructs 3D shapes from 2D observations without per-subject iterative optimization.

1 Introduction

  • Radiographs are practical for femoral reconstruction, but lost depth makes 3D recovery ill-posed despite its clinical value for planning, implant sizing, and follow-up.Distinct 3D shapes can produce nearly identical radiographs.Source: §1 · pp. 1–2
  • Conventional SSM reconstruction iteratively matches projected 3D features to radiographs, while single-dimensional models require recovering the 2D-to-3D mapping anew for each subject.Repeated projections across iterations and views make reconstruction time-consuming.Source: §1 · p. 2
  • The proposed joint 2D–3D SSM learns co-variation between 2D and 3D stationary velocity fields in a shared latent space.The model is trained from paired 2D and 3D shapes, with PCA applied to the concatenated fields.Source: §1 · pp. 2–3
  • For unseen shapes, reconstruction estimates latent coordinates from 2D observations and projects them onto the 3D subspace, avoiding per-subject iterative optimization.The method eliminates explicit 3D-to-2D projection at inference.Source: §1 · p. 3
  • On NMDID femoral data, the joint 2D–3D SSM outperforms a widely used traditional 3D SSM baseline.The introduction reports validation on NMDID but provides no metric values in these passages.Source: §1 · p. 3

2 Methods

The method builds a joint 2D–3D shape model from paired views and uses observed 2D segmentations to estimate unseen subjects’ 3D shapes. Joint PCA captures shared variation, while reconstruction estimates latent coordinates and maps them to a 3D deformation.

2.1 Joint 2D-3D Statistical Shape Model

  • The joint model represents paired 2D and 3D anatomical variation with SVFs registered to a common 3D template and its corresponding 2D projections.For each subject, it uses AP and ML views alongside the 3D segmentation; PCA is applied to the concatenated, normalized view-specific fields.Source: §2 · p. 4, §2.1 · pp. 4–5
  • Each view’s flattened SVFs are z-score normalized and scaled by the inverse square root of their dimensionality so no view dominates joint variance.PCA components approximate new multi-view shapes, with K selected to retain a desired percentage of total variance.Source: §2.1 · pp. 4–5

2.2 Patient-Specific Shape Reconstruction

  • For unseen subjects, 2D observations are used to estimate regularized latent coordinates, which are projected onto the 3D subspace to reconstruct the 3D shape.The latent coordinates are obtained by solving a closed-form linear system; the resulting 3D velocity field is denormalized, exponentiated, and applied inversely to the template.Source: §2.2 · pp. 5–6

3 Results

The experiments evaluate femoral reconstruction from standard AP and ML views on NMDID, comparing the joint model with a 3D-only SSM and examining the learned latent space. The joint model improves reconstruction metrics and inference time while retaining compact, coherent anatomical variation.

3.1 Dataset and Preprocessing

  • The 1,368 CT scans came from 781 individuals aged 7–97; subjects were split into registration, shape-model, validation, and held-out test sets.Right femurs were mirrored, aligned and cropped; AP and ML DRRs were generated and binarized as 2D segmentations.Source: §3.1 · p. 6, 2D and 3D Segmentations · p. 6

3.2 Experimental Setup

  • The joint SSM jointly decomposes 3D, AP, and ML stationary velocity fields using PCA, retaining 95% of variance with 38 components.The 3D-only baseline uses the same 3D velocity fields but no 2D training information, retaining 95% with 39 components.Source: Joint 2D–3D SSM · p. 7, Baselines · p. 7
  • Evaluation uses DSC, HD95, and MSD for held-out 3D femur reconstruction, plus average reconstruction time.The mean shape serves as a lower bound because it is returned regardless of input and uses no 2D observations.Source: Evaluation Metrics · p. 7, Baselines · p. 7

3.3 3D Reconstruction Evaluation

  • 96.15±1.65% vs 94.96±2.06% DSC, 0.70±0.28 vs 0.89±0.35 mm MSD, and 1.72±1.00 vs 2.24±1.22 mm HD95; inference took 2.76±0.09 vs 11.08±0.90 s.The comparisons are joint 2D–3D SSM versus the 3D SSM baseline on the held-out test set.Source: Table 1 · p. 8, §3.3 · p. 8
  • Reconstructions are predominantly within 1 mm of ground truth, with the strongest gains at the condyles, a key surgical-planning reference.Figure 2 compares surface distance for the mean shape, 3D SSM, and joint model across three representative test femurs.Source: §3.3 · pp. 8–9, Figure 2 · p. 8

3.4 Qualitative Analysis of the Latent Space

  • The joint model’s first three modes explain 30.3%, 20.9%, and 10.8% of variance, and 38 components capture 95%, close to the 3D SSM’s 39.The modes show coherent variation across 2D and 3D views; their 3D shapes are visually similar to the 3D SSM modes.Source: §3.4 · pp. 9–10, Figure 3 · p. 9

4 Conclusion

The paper presents a joint 2D–3D SSM that recovers 3D shape from 2D segmentations through closed-form inference. On NMDID, it outperforms a traditional 3D SSM and is roughly four times faster, while assuming known, fixed calibration.

4 Conclusion

  • The joint 2D–3D SSM encodes co-variation between 2D and 3D segmentations in a shared latent space.At inference, 3D shape is recovered from 2D segmentations without iterative optimization.Source: §4 · p. 10
  • On NMDID, the method outperforms the traditional 3D SSM on all reported reconstruction metrics and completes inference in 2.76 s, roughly four times faster.The joint model captures meaningful, compact anatomical variation with a 2D-consistent structure.Source: §4 · p. 10
  • The method assumes known, fixed calibration; extending it to uncalibrated settings is proposed as future work.The authors identify varying patient positioning and unavailable calibration data as settings where broader applicability would matter.Source: §4 · p. 10

Related papers

Loading 2609.09010v1…