Observation-Conditioned Latent Energy Priors for Sparse Implicit Neural Shape Completion

Paul Büschl; Ezequiel de la Rosa; Julia Wolleb; Julian McGinnis; César Nombela-Arrieta; Bjoern Menze

Source-linked AI summary

Observation-Conditioned Latent Energy Priors for Sparse Implicit Neural Shape Completion

Paul Büschl, Ezequiel de la Rosa, Julia Wolleb, Julian McGinnis, César Nombela-Arrieta, Bjoern Menze

arXiv:2609.03694v1Sep 3, 2026cs.CV

TL;DR

Sparse autodecoder inference can fit limited SDF observations while drifting toward implausible unseen geometry. This paper adds a post-hoc observation-conditioned energy to a frozen INR decoder, improving over global latent priors across the reported evaluations and supporting an observation-specific contribution.

Problem
Underconstrained sparse observations allow latent codes to fit measurements while decoding implausible unobserved geometry, a failure mode called latent drift.
Method
A post-hoc energy scores standardized latents conditioned on a permutation-invariant sparse-observation encoding and complements a validation-selected L2 prior during frozen-decoder inference.
Results
The conditional-energy objective improves over tuned L2 in the sparsest nucleus regimes and outperforms L2 and GMM K6 across reported MedShapeNet readouts.
Takeaways & Limitations
Matched observation context improves over shuffled context at every tested sample count, supporting an observation-specific contribution beyond a general latent-prior effect.
Takeaways & Limitations
Evaluation covers one controlled cell-nucleus SDF setting and one public MedShapeNet-derived SDF setting, leaving broader generalization and additional structured priors for future work.

Abstract

from arXiv · show

Implicit neural representations (INRs) can model continuous 3D shapes with a shared coordinate decoder and per-instance latent codes. At test time, autodecoder-style models commonly freeze the decoder and optimize a new latent code from sparse off-grid SDF samples. When these samples underconstrain inference, the latent can drift toward regions that fit the observations but decode implausible unobserved geometry. We propose a post-hoc observation-conditioned latent energy prior for frozen INR decoders. The energy scores standardized latents conditioned on a permutation-invariant encoding of the sparse observation set and is used as a residual expert alongside an L2 latent prior selected on validation data. We evaluate on a controlled cell-nucleus SDF dataset and a public MedShapeNet-derived SDF completion dataset. The proposed L2 objective augmented with conditional energy improves consistently over a validation-selected L2 baseline in the sparsest cell-nucleus regimes and, on MedShapeNet, outperforms both L2 and a six-component GMM latent-density prior across all reported readouts. A shuffled-context ablation is consistently weaker than matched context, supporting an observation-specific contribution. These results suggest that lightweight conditional energies can make pretrained INR decoders more observation-aware without retraining.

1 Introduction

Latent-conditioned INRs enable sparse continuous shape completion, but underconstrained test-time optimization can produce plausible fits with implausible unseen geometry. The paper introduces a post-hoc observation-conditioned energy prior for frozen decoders and evaluates it against lightweight global priors.

Sparse latent optimization can fit observed SDF samples while drifting toward codes that decode implausible unobserved geometry.
Global L2 and density priors regularize latent plausibility but score codes independently of the particular sparse observation set.L2 provides stable scale control, while stronger global priors capture latent-distribution structure without conditioning on measurements.
The proposed post-hoc conditional energy is trained from frozen-decoder latents paired with subsampled coordinate/SDF observations and acts as an observation-aware residual prior.Its observation encoding is permutation-invariant, allowing unordered sparse sets to condition the latent preference.
The method is evaluated on a controlled cell-nucleus SDF testbed and a public heterogeneous MedShapeNet mesh/SDF completion track.
The study identifies latent drift, introduces the conditional energy alongside tuned L2 for frozen-decoder inference, and compares it with global priors and context ablations.

2 Method

The method adds a post-hoc observation-conditioned energy prior to frozen-decoder latent inference, combining it with an L2 prior to guide sparse SDF completion. The energy is trained from matched latent–observation pairs and permutation-invariant contexts, then used as a differentiable test-time prior.

Inference: The workflow freezes a pretrained INR decoder and optimizes only the latent code using observation loss, an L2 prior, and conditional energy.The energy and context encoder are trained after decoder fitting, while test-time inference keeps both decoder and energy fixed.
Prior construction: The base L2 prior standardizes latents using training-set mean and scale, controlling latent magnitude while its validation-selected weight absorbs constant normalization factors.The conditional energy serves as an observation-aware residual expert alongside this global scale-control term.
Observation-conditioned prior: The observation context encodes unordered coordinate/SDF pairs with permutation-invariant pooling, and the energy scores standardized latents conditioned on that context.The context encoder maps each coordinate–SDF pair through an embedding, pools the set, and produces a context vector for the scalar energy network.
Energy training: The conditional energy is trained post hoc from training latents paired with sparse observation sets subsampled from the same shapes.Training samples observation counts from dataset-specific values and uses only train-split latents and observations.
Energy training: Energy Matching regresses gradients along interpolations from Gaussian noise to matched standardized training latents, conditioned on their observation contexts.Ranking losses distinguish matched from mismatched latent–context pairs, while off-manifold negatives are calibrated to have higher energy than matched pairs.
Energy training: The full energy-training loss combines gradient matching, ranking, negative calibration, and scale terms, while interpolants and Langevin-refined negatives are excluded from test-time inference.At inference, the learned energy is used as a differentiable prior rather than recomputing training-only samples.

3 Experiments

The experiments evaluate sparse SDF completion on controlled cell-nucleus and public MedShapeNet-derived datasets using shared metrics, baselines, and locked-test protocols.

Datasets and sparse observations: The evaluation covers cell-nucleus and MedShapeNet-derived SDF completion with 16–128 and 16–256 observed samples, respectively.The nucleus split has 2,924/699/492 train/validation/locked-test nuclei; MedShapeNet uses six anatomical categories with a balanced 960/120/120 split.
Metrics: Held-out SDF error is primary, with Grid SDF MAE, occupancy Dice/IoU, and nucleus surface Dice as secondary readouts.Surface Dice uses a fixed tolerance of 0.1 normalized coordinate units.
Baselines and protocol: All methods share the frozen decoder, sparse observations, latent initialization, optimization budget, and evaluation points within each setting.Baselines include No prior, validation-selected Tuned L2, and a six-component global GMM; the main method is L2 + Conditional Energy.
Nucleus evaluation: Table 1 reports nucleus locked-test performance for Conditional Energy alone and the combined L2-plus-Conditional-Energy method.The table caption abbreviates Conditional Energy as CondE and the combined method as Ours.
MedShapeNet evaluation: Table 2 reports MedShapeNet locked-test performance for the six-component global mixture and the combined L2-plus-Conditional-Energy method.GMM K6 is the six-component global latent mixture baseline.

4 Results

The combined L2 and Conditional Energy method performs best or tied-best on nuclei and leads MedShapeNet across reported readouts, while matched context improves over shuffled context.

Locked-test performance: On the locked nucleus test set, L2 + Conditional Energy is best or tied-best across sparse observation settings, while Conditional Energy alone remains below Tuned L2.The ablation improves over No prior, supporting use of the conditional energy as a complement to L2 rather than a replacement.
Locked-test performance: On MedShapeNet, L2 + Conditional Energy outperforms Tuned L2 and GMM K6 across all reported readouts and observed-sample counts.The global GMM often improves over Tuned L2, but the observation-conditioned method remains best across the reported MedShapeNet settings.
Paired gains over global priors: At 16–64 nucleus samples, gains over Tuned L2 are significant for all four metrics, whereas at 128 samples only surface Dice remains significant.The largest corrected p-value at 16, 32, and 64 samples is 1.38×10−2; surface Dice at 128 samples has p = 8.0×10−4.
Context ablation: On MedShapeNet, matched context improves Grid SDF MAE over shuffled context by 0.0026–0.0072 at every observed-sample count.Shuffled context still beats Tuned L2 on Grid SDF MAE, indicating both a general latent-prior component and an observation-specific contribution.
Qualitative examples: Qualitative reconstructions compare ground truth, Tuned L2, GMM K6, and L2 + Conditional Energy across anatomical categories.The reported comparison emphasizes preservation of category-compatible shape structure under identical sparse observations.

5 Discussion & Conclusion

Observation conditioning is most useful when sparse inference is underconstrained: matched conditional energy complements tuned L2 and improves over global priors on heterogeneous MedShapeNet. The limited evaluation supports observation-aware post-hoc priors while leaving broader datasets, priors, and observation patterns for future work.

Results: Matched context improves over shuffled context at all tested observation counts, supporting an observation-specific contribution beyond a general latent-prior effect.Tuned L2 provides stable global shrinkage, while conditional energy acts as an observation-conditioned complement.
Interpretation: Observation conditioning is most useful when sparse test-time inference is underconstrained, where a global isotropic L2 preference can be too coarse.This interpretation is consistent with the stronger structure and larger nearest-neighbor distances observed in MedShapeNet latents, but the latent comparison alone does not establish the hypothesis.
Results: Matched conditional energy improves on GMM K6 across the reported MedShapeNet readouts, suggesting information beyond a global multimodal latent prior.The latent-space comparison finds stronger category structure and larger nearest-neighbor distances for MedShapeNet than for nucleus latents.
Limitations: The study covers one controlled cell-nucleus SDF setting and one public MedShapeNet-derived SDF setting, limiting the breadth of its evaluation.Broader context randomization, out-of-distribution tests, and additional structured priors remain future work.
Conclusion: The authors suggest lightweight post-hoc priors can make pretrained INR decoders more observation-aware without retraining.Future work includes stronger conditional energies, realistic partial-surface observations, broader INR tasks, and topology-rich reconstruction.