PaperlayerView exact revision on arXiv

Source-linked AI summary

Plausible but Unimplantable: A Trustworthiness Audit of Shape Completion Foundation Models for Cranial Defect Reconstruction

Marek Wodzinski, Kamil Kwarciak, Daria Hemmerling, Mateusz Daniol

arXiv:2610.30035v1cs.CV

TL;DR

Cranial implants are still designed by hand, and shape completion models appear to offer a way to automate that work. This audit tests released pretrained models on cranial defects against retained bone and a geometric implantability criterion, finding that none of the audited outputs could be put in a patient.

  • Problem

    Cranial implants remain slow, specialist-designed CAD work, while resemblance-based completion scores do not establish whether predictions add material beyond retained bone or physically fit a patient.

    Source: §1 · p. 2

  • Method

    The authors audit three released pretrained shape completion models zero-shot on cranial defects, comparing outputs with retained bone and screening for collision-free placement with nonnegative advantage over that baseline.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    None of the model outputs could be put in the patient, despite some appearing to restore the skull’s shape.

    Source: §5 · p. 8

  • Takeaways & Limitations

    At the tested point densities, emitted surfaces do not encode recoverable implant thickness, and the released models are not trustworthy for this clinical use.

    Source: §5 · pp. 8–9

    The audit covers released checkpoints used as-is; cranial fine-tuning is untested, and only 29 real defects were evaluated.

    Source: §5 · p. 9

Abstract

from arXiv · show

A patient who has lost part of the skull vault needs an implant that fits the defect exactly, and such implants are still drawn by hand in CAD by specialists, which makes each case slow and expensive. The task can be considered as a shape completion problem, so pretrained completion models, and now the first shape completion foundation models, look like a way to remove that manual step. We audited three publicly released pretrained shape completion models, two of them foundation models, applied zero-shot to 239 cranial defects, some carved from public skull datasets and some real post-craniectomy cases. Each output was compared with simply returning the patient's own remaining bone, and tested for whether the predicted material could be placed without intersecting the bone that is still there. On the 210 carved defects none of the three improved on doing nothing, and on the 29 real cases only one did so beyond chance. Yet not one output, from any model in any configuration, was implantable: none managed both to stay clear of the bone the patient still has and to place more material than doing nothing, where a task-specific volumetric model, trained for this task and selected on these cases, was implantable on all but one. The output that most convincingly resembles a skull is no more placeable than the rest, and at these point densities implant thickness cannot be read off the emitted points at all, which is a property of the output representation rather than of any one model. On this evidence the shape completion foundation models we could obtain cannot be trusted and, used as released, are not usable in clinical practice. Substantial research towards trustworthy foundation models for shape completion is still needed.

1 Introduction

Cranial implants remain slow, specialist-designed CAD work, while shape completion foundation models appear to offer a route to automate it. The audit evaluates whether their outputs meet clinical needs rather than merely resembling a reference.

1 Introduction

  • Hand-designed CAD implants make each case slow and expensive, limiting how many patients can be offered one.Implants are shaped to the patient’s anatomy after bone is removed from the skull vault.Source: §1 · p. 2
  • Foundation models for shape completion appear relevant because a skull defect is a partial shape, and a checkpoint used as released could remove training and manual design steps.The first shape-completion foundation models are beginning to appear.Source: §1 · p. 2
  • Reference-shape resemblance cannot establish whether a model adds material beyond retained bone or whether its prediction can physically fit the patient.These are distinct from the question of whether a completed shape looks like a reference.Source: §1 · p. 2
  • The audit compares three publicly released pretrained models, applied zero-shot, against the patient’s retained bone and tests whether predicted material can be placed without intersecting it.Its acceptance criterion asks whether the output could be put in a patient.Source: §1 · p. 2

2 Related Work

Prior cranial implant work primarily frames design as volumetric completion and evaluates reference similarity. The audit emphasizes that this benchmark framing does not directly answer whether a model’s output can be used for an implant.

2 Related Work

  • Cranial implant completion benchmarks cover synthetic SkullBreak and SkullFix defects and real MUG500+ craniectomy cases, setting a standard for zero-shot checkpoints.Networks trained for the task perform well on the carved defects.Source: §2 · p. 2
  • The audit includes two models offered or advertised at foundation scale and AdaPoinTr as a generalist control, trained on everyday ShapeNet-55 objects without skulls.MSN-FM uses text conditioning; MGPC adds an image branch, though its released code requires an image at inference.Source: §2 · p. 3
  • Reference similarity benchmarks poorly match clinical decisions: Chamfer distance is sensitive to density and outliers, and synthetic-to-real performance can degrade.Without a do-nothing baseline, a benchmark cannot distinguish a model from the trivial solution.Source: §2 · p. 3

3 Materials and Methods

The study evaluates pretrained models on synthetic and real defects using a retained-bone baseline and a geometric implantability screen. It also controls for point allocation, model selection, and protocol choices that could affect the comparison.

3 Materials and Methods

  • 239 cases comprise 210 synthetically carved SkullBreak and SkullFix defects and 29 real MUG500+ craniectomy defects.The real cases include manufactured implants, so their results assess whether material went where the implant goes rather than reconstruction accuracy.Source: §3 · p. 3
  • Models receive the retained portion of one sampled skull surface; recall at 2 mm is reported as the per-skull advantage over returning retained bone alone.The no-op baseline matters because the removed bone is a thin shell, so retained bone supplies coverage for free.Source: §3 · p. 3
  • The study reports both common 6144-point and native-budget results because point allocation affects recall and penetration in opposing directions.A uniform whole-skull resample has a median per-skull advantage of +0.169; the two scoring conventions agree on ordering but flip AdaPoinTr’s sign (+0.013 native, −0.036 common).Source: §3 · pp. 3–4
  • An output is implantable only if it is collision-free at 1 mm and has nonnegative advantage over the no-op at 2 mm.This geometric screen is necessary, not clinical approval; the deployment frame uses only the defective skull.Source: §3 · p. 4
  • A task-specific volumetric model provides a reference ceiling, but it was selected by Dice on the same 210 carved test cases on which it is reported.It was trained on disjoint training splits, and its results on the 29 real defects are out of domain.Source: §3 · p. 4
  • The robustness checks sweep six protocol dimensions and add a positive control that gives each model the complete skull.Checkpoint validation differed: MSN-FM and MGPC were checked against upstream behavior, while AdaPoinTr’s check only confirmed that weights loaded.Source: §3 · p. 4

4 Results

Across the 239 skulls, none of the three pretrained models produced an implantable output, and none improved over the retained-bone baseline on the carved defects. Results also show that familiar coverage scores and skull-like appearance do not establish clinical usefulness; at the tested point densities, implant thickness is not represented in the output.

4 Results

  • MSN-FM recalled 0.462 of the removed region at 5 mm on carved defects versus 0.439 on its compact training defect, but coverage alone cannot show whether a model added anything.A no-op recovers 0.049 of the compact defect and 0.783 of the dispersed defects in the skull datasets at 5 mm.Source: §4 · pp. 4–5
  • At 6144 points, all three models had negative median advantage over a no-op: AdaPoinTr −0.036, MGPC −0.130, and MSN-FM −0.156.The task-specific reference reached +0.745 on an allocation that was not like-for-like with the completion models.Source: §4 · p. 5, Table 1 · p. 5
  • On the 29 real craniectomy defects, MGPC reached +0.118 advantage over a no-op on 24 of 29 skulls, while AdaPoinTr’s +0.021 was not established.The clinical p-values for AdaPoinTr disagree between the sign test and Wilcoxon signed-rank test; every synthetic result was significant at p < 10−24 under both tests.Source: Table 2 · p. 5, §4 · p. 6
  • 0 of 239 skulls were implantable for every audited model, versus 238 of 239 for the task-specific reference; none of 2,868 model-arm-skull combinations was collision-free at 1 mm.The reference was test-selected; a ground-truth control using the same point allocation collided on none of the 239 skulls.Source: Table 1 · p. 5, §4 · p. 6, Figure 1 · p. 6
  • At the tested point densities, implant thickness cannot be read from emitted points; 11 of 29 real implants (38 %) are thinner than the 3 mm design target.At comparable 2.4 mm spacing, 0.447 of neighborhoods on a 3 mm plate include points from the opposite face; the passage attributes the failure to representation, not measurement.Source: §4 · pp. 6–7, Figure 3 · p. 7
  • On the B0001 example, all three outputs intersect retained bone, although two exceed the no-op; the medians in Table 1 do not.AdaPoinTr’s points lie on a surface, unlike the solid-cap appearance of AdaPoinTr and MGPC or the granular MSN-FM addition.Source: Figure 2 · p. 7
  • Prompt changes were small relative to each foundation model’s no-information floor, and 97.5 % of MSN-FM’s text code was exactly zero.The text channel was described as decorative in MGPC and for all but one prompt in MSN-FM.Source: §4 · pp. 7–8
  • Across six protocol dimensions, no audited row was implantable; AdaPoinTr’s 16,384-point checkpoint was collision-free on 1 of 239 skulls and had advantage −0.198.That checkpoint was the only one to resolve a 3 mm plate; with the complete skull as input, AdaPoinTr reached +0.042 over no-op.Source: Figure 3 · p. 7, §4 · p. 8

5 Discussion

The released completion models fail the practical demands of implant design: their outputs can look convincing without being physically placeable, and their point-surface representation cannot recover thickness. Repair would still require patient-specific manual work, while the audit leaves finetuning and clinical evaluation untested.

5 Discussion

  • 5.40 to 5.93 mm is the reconstructed thickness for every model on every cohort, against references spanning 3.05 to 7.25 mm.The measured value comes from the shared 3 mm reconstruction radius, not from thickness encoded in the model outputs.Source: §5 · p. 8
  • Two barriers are structural: training defects remove 10.8 times as many surface points (Cliff’s δ = 0.83), and point surfaces cannot recover thickness at this density.The authors state that neither barrier would be resolved by changing the loss weight or adding more data.Source: §5 · p. 8
  • Outputs that look like skulls may still be unplaceable; a clean cap can cut into retained bone without an obvious visual warning.The authors identify this visually reassuring failure as the harder case to catch.Source: §5 · p. 8
  • Surfacing, trimming, and thickening require per-patient thresholds set by hand, and repairing the outputs is unlikely to be quicker than CAD design.A pipeline that needs this manual work does not remove the step the pretrained model was meant to replace.Source: §5 · pp. 8–9
  • Thickness recovery requires volumetric or implicit outputs, or point spacing below half the plate thickness; physical acceptance must also enter the objective.The authors also call for training defects that match clinical defects and a do-nothing baseline for overlap scores.Source: §5 · p. 9
  • The audit covers released checkpoints used as-is, has no clinician assessment, and includes only 29 real defects, so it does not settle a clinical question.Finetuning on cranial data and task-specific point-cloud methods were not tested.Source: §5 · p. 9

Related papers

Loading 2610.30035v1…