Source-linked AI summary
MedBOE: Optimization-Based Estimation of Complete Patient Body Models from Whole-Body Medical Imaging
Hoijoon Jung, Yuhao Wei, Mingxiao Tu, Younhyun Jung, Michael Fulham, Jinman Kim
TL;DR
Whole-body medical images can omit regions needed for complete patient body models, limiting analyses that use those models. MedBOE uses an SMPL-based, automated, training-free optimization pipeline to estimate missing regions. It performed best overall in volumetric accuracy, ranked close second geometrically, and derived BHMs within clinically accepted tolerances; validation on real clinical incompleteness remains needed.
Problem
Whole-body medical images can omit body regions, while prior approaches had not addressed complete external body-model estimation from such routine scans.
Source: §1 · pp. 2–3Method
Results
MedBOE achieved the highest volumetric accuracy, ranked close second in geometric accuracy, and derived BHMs with low percentage errors.
Source: §6 · p. 19Takeaways & Limitations
MedBOE provides an automated, training-free approach for estimating complete patient body models, with derived BHMs reported within clinically accepted tolerances.
Source: §7 · p. 23Because evaluation primarily used virtually trimmed models, validation on clinical acquisitions with real field-of-view incompleteness is needed to clarify generalization.
Source: §6.1 · p. 22, §7 · p. 23
Abstract
from arXiv · showhide
True whole-body medical imaging captures a patient from head to toe and provides comprehensive anatomical and/or functional information. This coverage enables reconstruction of the external body surface, yielding patient body models. However, scanner geometry (e.g., bore diameter), patient characteristics, and positioning can leave body regions missing (e.g., lateral arms/torso, hands, forearms, elbows), producing incomplete patient body models. Such incompleteness limits downstream medical analyses, including derivation of body-habitus metrics (BHMs), such as body volume and surface area, for injected radiotracer activity regimens, cardiometabolic risk assessment, and computational anatomical modeling for medical imaging simulation and radiation dosimetry. We present the Medical Body model Optimization-based Estimator (MedBOE), an automated, training-free framework for estimating complete patient body models from whole-body images with missing body regions. Our innovation is to leverage the Skinned Multi-Person Linear (SMPL) template, a parametric body mesh representing a broad range of body shapes and poses, in medical imaging. MedBOE aligns the SMPL template to the incomplete patient body models via attribute-guided optimization and estimates missing body regions through geometrically and volumetrically constrained deformation. We evaluated MedBOE using two public datasets: AutoPET (91 whole-body PET-CT images) and Human Implicit Tissues (HIT; 40 whole-body segmentations from MRI images). MedBOE achieved the best overall volumetric and geometric performance among evaluated methods. BHMs derived from MedBOE were within 2% error, suggesting low percentage errors for downstream analyses. MedBOE offers an automated, training-free method for estimating complete patient body models from whole-body images with missing body regions.
1 Introduction
Whole-body imaging can provide external body models and body-habitus metrics (BHMs), but scanner and patient factors often leave regions missing. MedBOE addresses the resulting gap with an automated, training-free pipeline for estimating complete patient body models.
1 Introduction
- Whole-body imaging supports reconstruction of external body surfaces and BHMs used in radiotracer dosing, cardiometabolic risk assessment, and anatomical modeling for simulation and dosimetry.Surgical navigation is another stated application of patient body models.Source: §1 · p. 2
- Scanner field-of-view and axial coverage, patient size, and positioning can leave arms, torso, hands, forearms, or lower limbs missing, reducing model utility for downstream analyses requiring accurate BHMs.Source: §1 · p. 2
- Multimedia completion methods typically use non-clinical upright poses and supervised training data unavailable for medical imaging, while clinical scans may omit entire regions.These differences limit direct transfer to clinical settings.Source: §1 · pp. 2–3
- Prior medical studies addressed image-domain truncation or SMPL fitting for other aims, but the authors found no prior work estimating complete body models from routine whole-body medical images with missing regions.Source: §1 · pp. 3–4
- MedBOE combines image-derived incomplete models and landmarks, attribute-guided SMPL fitting, and missing-region estimation with volume-constrained deformation; it achieved the best overall performance among evaluated methods, with derived BHMs within 2% error.Evaluation used AutoPET and HIT, and the source code and optimization weights are publicly available.Source: §1 · p. 4
2 Related Work
Related work spans deep-learning shape completion, parametric template fitting, and medical-imaging approaches to truncation or SMPL fitting. The paper distinguishes MedBOE by targeting continuous complete external body models from routine whole-body scans with missing regions.
2.1 3D Shape Reconstruction Methods
- 3D shape-reconstruction methods use deep learning to complete partial inputs, but clinical use faces scarce paired clinical data and scans with entirely absent regions.Examples include PCN, IF-Net, and ShapeFormer.Source: §2.1 · p. 5
2.2 Parametric Body Template Mesh Fitting Methods
- Parametric fitting methods align a template such as SMPL to incomplete inputs, but non-clinical pose and coverage assumptions—and temporal requirements in sequence-based methods—limit direct medical-imaging applicability.Examples include IP-Net, ArtEq, and motion-aware fitting methods.Source: §2.2 · p. 5
2.3 Image Compensation and SMPL-Based Body Model Fitting in Medical Applications
- Medical truncation-compensation methods recover image or attenuation-map information rather than continuous complete external body surfaces.Source: §2.3 · p. 5
- Medical SMPL studies have inferred body composition or internal anatomy, generally relying on complete coverage or semi-manual fitting; MedBOE instead estimates complete external surfaces and complements image-domain truncation compensation.Source: §2.3 · p. 6
3 Method
MedBOE reconstructs incomplete body surfaces from whole-body images, fits an SMPL template, then estimates and deforms missing regions to produce a complete patient model. Its staged optimization uses anatomical, geometric, and volumetric constraints.
3 Method
- The three-module pipeline reconstructs an incomplete model with landmarks, fits an SMPL template, then estimates missing regions and deforms the stitched model into a final complete surface.Source: §3 · p. 6
3.1 SMPL Template Mesh
- The SMPL template represents the body as a 6,890-vertex triangulated mesh parameterized by shape, pose, and global translation.Source: §3.1 · pp. 6–7
3.2 Body Model Reconstruction Module
- Body reconstruction segments whole-body images, converts masks into smoothed and simplified incomplete meshes, extracts patient metadata, and automatically annotates sparse and dense virtual-marker landmarks.Sparse landmarks are included in fitting only when confidence exceeds 0.5.Source: §3.2 · p. 7
3.3 Global Parametric Fitting Module
- Global fitting jointly optimizes shape, pose, and translation using landmark and surface alignment, pose priors, patient-weight consistency, and a self-intersection penalty.The parameters are initialized to zero and optimized for 500 iterations.Source: §3.3 · p. 7
3.4 Missing Body Region Estimation and Surface Deformation Module
- Coarse deformation fits the SMPL mesh to observed surfaces while preserving local geometry, then trims and stitches estimated missing regions; fine deformation smooths discontinuities using surface, normal, and volume constraints.Fine deformation is restricted in forearms, hands, and feet to overlapping regions to avoid propagating discontinuities.Source: §3.4 · pp. 8–9
3.5 Implementation
- Implementation uses VTK and Deep Virtual Marker for reconstruction, and PyTorch with smplx and Kaolin for fitting and deformation; geometric-term weights are bounded to maintain stability because higher weights caused mesh collapse.Source: §3.5 · pp. 9–10
4 Experimental Setting
The evaluation uses two datasets with simulated incompleteness and compares model accuracy, derived body-habitus metrics, runtime, and baseline methods. The setup also includes an ablation of the missing-region module and specified statistical tests.
4.1.1 AutoPET Dataset
- AutoPET evaluation retained 91 of 1,014 examinations after exclusions and used five-fold cross-validation; incomplete models were simulated by trimming PET-derived ground-truth meshes to approximate CT field-of-view loss.The trimming removed on average 1.75% of male and 1.32% of female body volume.Source: §4.1.1 · p. 10
4.1.2 HIT Dataset
- HIT provides complete MRI segmentations and aligned SMPL meshes; evaluation used predefined splits and simulated trimming, with estimated weights because measured weights were unavailable.Trimming removed on average 3.35% of male and 2.38% of female body volume.Source: §4.1.2 · pp. 10–11
4.2.1 Accuracy of Estimated Complete Body Models
- Complete-model accuracy was assessed separately by gender using five metrics for size, volumetric overlap, and surface geometry; AutoPET scoring was limited to regions present in its PET-derived ground truth.HIT permitted evaluation over the entire body.Source: §4.2.1 · p. 11
4.2.2 Accuracy of BHMs Derived from Estimated Models
- Five body-habitus metrics—surface area, volume, and chest, waist, and hip circumferences—were compared with ground truth, with accuracy evaluation limited to HIT.The reported error measures were mean percentage and absolute errors.Source: §4.2.2 · pp. 11–12
4.3 Runtime Delay
- Runtime measured model estimation from an incomplete body model and excluded image segmentation and reconstruction of that input.Measurements used a desktop with an NVIDIA TITAN V GPU.Source: §4.3 · p. 12
4.4 Baselines
- MedBOE was compared with IF-Net, ShapeFormer, ArtEq, and PCN, with training and evaluation conditions adapted to dataset availability and method requirements.PCN volumetric metrics were excluded because its output lacks a surface representation; ArtEq was evaluated only on HIT.Source: §4.4 · p. 12
4.5 Ablation Study
- The ablation compared the full pipeline with a variant that omitted missing-region estimation and surface deformation while holding other settings constant.Source: §4.5 · p. 12
4.6 Statistical Analysis
- The analysis used one-way ANOVA with Tukey HSD, paired and Welch t-tests, effect sizes, and 95% confidence intervals where applicable, with significance set at p < 0.05.Source: §4.6 · p. 13
5 Results
MedBOE generally performed strongly on complete-body-model comparisons across AutoPET and HIT, with dataset- and metric-specific exceptions. Derived body-habitus metrics had low errors, but MedBOE required substantially longer runtime than the baselines; its missing-region estimation and deformation module improved most evaluated metrics.
5.1 Estimated Complete Body Models
- The figures compare estimated models with ground truth and highlight missing-region differences at elbows, forearms, and hands across methods.AutoPET figures show MedBOE versus IF-Net; HIT figures also mark a 10 cm above-head apex cutoff.Source: Figure 2 · p. 14, Figure 3 · p. 14, Figure 4 · p. 16
- AutoPET: MedBOE led male volumetric metrics and female RVE, while IF-Net led female CD, HD, RBE, and IoU; MedBOE–IF-Net differences were nonsignificant.Both methods significantly outperformed PCN and ShapeFormer in the reported gender-specific comparisons.Source: §5.1 · p. 13, Table 2 · p. 15
- HIT: MedBOE led male volumetric metrics and CD, and female volumetric results; PCN led male HD and female HD, while IF-Net led female CD.MedBOE significantly outperformed IF-Net in male RBE and RVE, and ArtEq in female RBE.Source: Table 2 · p. 15, §5.1 · pp. 16–17
- Across AutoPET and HIT, MedBOE had smaller absolute mean differences than IF-Net in RBE, RVE, CD, and HD, and than ShapeFormer in RBE, RVE, IoU, and CD.MedBOE had no significant cross-dataset differences in RBE or RVE, but did in IoU, CD, and HD.Source: §5.1 · p. 17
5.2 Body Habitus Metrics
- On HIT, MedBOE-derived BHM percentage errors were below 1% for body volume and hip circumference, about 1.6% for waist circumference, and around 2% for surface area and chest circumference.The reported absolute mean errors were 0.001±0.000 m3 for volume and 0.042±0.031 m2 for surface area.Source: §5.2 · p. 17, Table 3 · p. 18, Figure 5 · p. 18
5.3 Runtime Delay
- MedBOE required 1254.07 ± 184.47 seconds per complete patient body model (813.56–1808.82), significantly longer than ArtEq, PCN, IF-Net, and ShapeFormer (Tukey’s HSD, all p < 1.00 × 10−12).Source: §5.3 · p. 18
5.4 Ablation Study
- Adding missing-region estimation and surface deformation significantly improved all evaluated metrics except HIT Hausdorff distance, where no significant difference was observed.The ablation comparison is shown in Table 4.Source: §5.4 · p. 19, Table 4 · p. 19
6 Discussion
MedBOE performed strongly on volumetric metrics and maintained performance across genders and datasets, while geometric results were more mixed. The discussion also identifies constraints from SMPL fitting, evaluation design, and landmark annotation.
6 Discussion
- RBE ≤2.15%, RVE ≤1.01%, and IoU ≥0.92; MedBOE generally led in volumetric metrics, with exceptions for RBE and IoU on AutoPET female models.RBE and RVE were lower than the proportions of trimmed body regions, indicating effective estimation of missing regions.Source: §6 · pp. 19–20
- MedBOE's accurate missing-region estimation drove volumetric gains, significantly outperforming second-best methods except for specified HIT metric comparisons.The missing-region estimation and surface-deformation module significantly improved performance in ablations (paired t-tests, p < 0.05).Source: §6 · p. 20
- MedBOE ranked close to second in geometric accuracy, comparable to IF-Net on AutoPET and PCN on HIT, with marginal differences.Nearest-point metrics can favor gap-bridging patches over anatomically continuous surfaces, especially when missing regions are small.Source: §6 · p. 20
- Across genders and datasets, MedBOE showed no statistically significant gender differences and generally smaller dataset-dependent variation than baseline methods.Its performance remained relatively consistent on HIT despite larger missing regions than AutoPET.Source: §6 · p. 21
- MedBOE's BHM errors fell within cited clinically acceptable ranges, including 0.001 ± 0.000 m^3 in volume and 0.016 ± 0.012 m in waist circumference.The discussion cites up to 0.5 m^2 as an acceptable body-surface-area difference among commonly used clinical estimation formulas.Source: §6 · p. 21
- The case-wise optimization and deformation made MedBOE significantly slower than all baselines, trading computational efficiency for avoiding dataset- or gender-specific training.Runtime increased when EContact was enabled; measurements used an NVIDIA TITAN V because PCN was incompatible with newer GPUs in the study environment.Source: §6 · pp. 21–22
6.1 Limitations and Future Work
- MedBOE's fitting is constrained by multimedia-pretrained models and SMPL parameterization, while constrained fine deformation may leave some local regions incompletely aligned.The authors propose medical-imaging-specific fitting models and local deformation constraints as future directions.Source: §6.1 · p. 22
- Virtual trimming enabled controlled comparisons but cannot reproduce all clinical incompleteness; AutoPET metrics also exclude body regions absent from its PET-derived references.The study also lacked a publicly available full-body wide-transverse-FoV CT dataset and notes that contacting body parts can reduce landmark reliability and complicate fitting.Source: §6.1 · pp. 22–23
7 Conclusion
MedBOE achieved the best overall performance across volumetric and geometric metrics, with body-habitus metrics within clinically accepted tolerances; validation on clinical acquisitions with real field-of-view incompleteness remains needed. The appendix details its fitting and deformation attributes and illustrates simulated trimming and challenging estimates.
7 Conclusion
- MedBOE achieved the best overall performance across volumetric and geometric metrics, and its derived body-habitus metrics were within clinically accepted tolerances.Further validation using clinical acquisitions with real field-of-view-related incompleteness is needed to clarify generalization to clinical scenarios.Source: §7 · p. 23
A.1 Global Parametric Fitting Attributes
- Global parametric fitting combines landmark and surface alignment terms with pose, weight, self-intersection, and dense-landmark constraints.Sparse landmark distances are confidence-weighted, and only landmarks with confidence above 0.5 are included; surface correspondences are filtered by spatial and angular agreement, with an additional head–arm mismatch exclusion.Source: §A.1 · pp. 24–25
- The pose prior regularizes SMPL pose, while weight consistency matches estimated and reference body weight; the contact penalty targets intersecting vertices.The contact criterion treats vertices as intersecting when their Euclidean distance is below 2 cm while their geodesic distance exceeds 30 cm.Source: §A.1 · pp. 24–25
A.2 Missing Body Region Estimation and Surface Deformation Attributes
- Coarse deformation fits sampled surfaces while constraining edge lengths, Laplacian smoothness, and adjacent-face normal consistency.Surface-distance pairs are retained only when their Euclidean distance is at most 0.3 m and their surface-normal angular difference is below 90°.Source: §A.2 · pp. 25–26
- Fine deformation aligns the fitted SMPL mesh with the initial complete model while constraining surface orientation, smoothness, and volume.The volume term preserves the fitted SMPL mesh volume during deformation.Source: §A.2 · p. 26
Appendix B Incomplete Patient Body Models from Simulated Trimming and CT Images
- Simulated trimming approximated field-of-view-related incompleteness in routine PET-CT acquisitions by applying a transverse field-of-view constraint to PET-derived ground-truth models.The comparison shows external surfaces in blue and internal surfaces exposed by truncation in red.Source: Appendix B · p. 26, Figure 6 · p. 27
C.1 Zoomed Comparison of AutoPET Estimated Complete Patient Body Models
- AutoPET zoomed views compare methods around regions above the shoulders and legs, where full-body views showed visible differences.The views are intended to support comparison of surface continuity and local body-shape estimation around missing regions.Source: §C.1 · p. 27, Figure 7 · p. 28, Figure 8 · p. 28
C.2 Representative Challenging Case from AutoPET
- In a challenging AutoPET case, MedBOE estimated complete arm regions but with an inaccurate pose, whereas IF-Net mainly closed missing surfaces and ShapeFormer and PCN did not fully recover the arms.The arms lay alongside a large-volume patient’s body, making them difficult for all methods to distinguish from the trunk.Source: §C.2 · pp. 27, 29, Figure 9 · p. 29
C.3 Statistical Analysis of AutoPET Estimated Complete Patient Body Models
The AutoPET appendix reports pairwise method statistics and male–female comparisons, then uses zoomed HIT views and a challenging case to show local differences among estimates.
C.3 Statistical Analysis of AutoPET Estimated Complete Patient Body Models
- AutoPET pairwise method comparisons use Tukey HSD tests, with p-values, Cohen’s d, and 95% confidence intervals reported in Table 5.Table 5 is presented across two parts.Source: §C.3 · p. 29, Table 5 · pp. 29–30
- AutoPET male–female comparisons use Welch’s independent two-sample t-tests, reporting mean differences, Cohen’s d, and 95% confidence intervals as female minus male.The comparisons are presented in Table 6.Source: §C.3 · p. 30, Table 6 · p. 31
C.4 Zoomed Comparison of HIT Estimated Complete Patient Body Models
- HIT zoomed views cover hands, forearms, and legs, where anatomical cues were limited, to compare local differences between MedBOE and baseline estimates.The views include male and female examples; the figure caption identifies hands and forearms in the top row and legs in the bottom row.Source: §C.4 · p. 30
C.5 Representative Challenging Case from HIT
- In a challenging HIT case with missing hands and forearms, MedBOE estimated complete upper limbs but showed elbow-joining and heel artifacts, while other methods produced incomplete or collapsed estimates.PCN predictions were visually closer to ground truth in some regions but did not provide a continuous surface estimate.Source: §C.5 · p. 31
C.6 Statistical Analysis of HIT Estimated Complete Patient Body Models
The HIT appendix describes pairwise method statistics and male–female comparisons, alongside a visual comparison of regional estimates for a male example.
C.6 Statistical Analysis of HIT Estimated Complete Patient Body Models
- HIT pairwise method comparisons use Tukey HSD tests and report p-values, Cohen’s d, and 95% confidence intervals.Table 7 contains the pairwise comparison statistics.Source: §C.6 · p. 31, Table 7 · p. 32
- HIT male–female comparisons use Welch’s independent two-sample t-tests, with mean differences, Cohen’s d, and 95% confidence intervals reported as female minus male.The passage identifies these results as Table 8.Source: §C.6 · p. 31
C.4 Zoomed Comparison of HIT Estimated Complete Patient Body Models
- The HIT male visual comparison shows ground truth and four method estimates, with hands and forearms above and legs below.The methods shown are MedBOE, IF-Net, ShapeFormer, and PCN.Source: Figure 10 · p. 33
C.7 Statistical Analysis of Dataset-Wise Comparisons
Dataset-wise AutoPET–HIT comparisons report statistical estimates for each method and metric, with differences oriented as HIT minus AutoPET.
C.7 Statistical Analysis of Dataset-Wise Comparisons
- The dataset-wise comparisons cover each method and metric across AutoPET and HIT.Mean differences are defined as HIT minus AutoPET.Source: §C.7 · p. 33
- The reported dataset-wise comparison statistics include p-values and Cohen’s d.The passage also specifies 95% confidence intervals.Source: §C.7 · p. 33
- The mean-difference direction is HIT minus AutoPET, so positive and negative differences are oriented relative to that ordering.This describes the reported subtraction order, not the direction of any particular result.Source: §C.7 · p. 33
C.8 Statistical Analysis of Ablation Study
The ablation statistics compare results with and without the missing-region estimation and surface-deformation module. The supplied passages also identify dataset-wise and HIT male–female statistical comparisons.
C.8 Statistical Analysis of Ablation Study
- Table 10 compares results with versus without the missing-region estimation and surface-deformation module, reporting p-values, Cohen’s d, and 95% confidence intervals.Mean differences are calculated as with the module minus without it.Source: §C.8 · p. 33
- Table 8 reports HIT male–female comparison statistics from Welch’s independent two-sample t-tests.Source: Table 8 · p. 34
- Table 9 is identified as a dataset-wise statistical comparison between AutoPET and HIT.Source: Table 9 · p. 34
Appendix D Results of Missing Region Estimation
MedBOE achieved the strongest missing-region estimation performance across the reported datasets, with one HIT metric exception. The results also note excluded method outputs and show representative HIT estimates.
Appendix D Results of Missing Region Estimation
- MedBOE performed best across all metrics for male and female AutoPET models; ShapeFormer ranked second volumetrically and IF-Net geometrically.Four IF-Net cases and 12 ShapeFormer cases were excluded after valid-output generation failed.Source: Appendix D · p. 35, Table 11 · p. 37
- On HIT, MedBOE led across metrics except female CD, where PCN performed best; ShapeFormer ranked second volumetrically and IF-Net in HD.ArtEq’s severe self-overlap prevented watertight-surface computation, so its RVE and IoU results were excluded.Source: Appendix D · p. 35, Table 12 · p. 37
- A representative challenging HIT case compares the GT and incomplete models with outputs from MedBOE, IF-Net, ShapeFormer, ArtEq, and PCN.Source: Figure 12 · p. 36
MedBOE
- Table annotations mark statistical differences from second-best results with an asterisk and identify methods with excluded failure cases using a circumflex.Source: MedBOE · p. 37