PaperlayerView exact revision on arXiv

Source-linked AI summary

CheXmask-U: Quantifying uncertainty in landmark-based anatomical segmentation for X-ray images

Matias Cosarinsky, Nicolas Gaggion, Rodrigo Echeveste, Enzo Ferrante

arXiv:2512.10715v2cs.CV

TL;DR

Landmark-based anatomical segmentation lacks established uncertainty estimates, while pixel-based segmentation can produce anatomically implausible predictions. This paper derives latent and node-wise predictive uncertainty from a variational CNN–graph model, finding that these signals respond to perturbations and support out-of-distribution detection. It also releases a large dataset of landmark segmentations with per-node uncertainty estimates.

  • Problem

    Pixel-based segmentation can produce anatomically implausible predictions, and prior uncertainty estimation has largely focused on pixel-level segmentation rather than landmark-level confidence.

    Source: §1 · p. 2, §5 · p. 12

  • Method

    The authors use a variational CNN–graph model to estimate uncertainty from latent distribution parameters and from stochastic landmark predictions sampled from the latent posterior.

    Source: §1 · p. 3, §3.2 · p. 5

  • Results

    Both uncertainty measures respond to image perturbations and indicate out-of-distribution chest X-rays, while providing complementary spatially informative signals with lower computational overhead than established pixel-level methods.

    Source: §5 · p. 12

  • Takeaways & Limitations

    CheXmask-U provides 657,566 chest X-ray landmark segmentations with per-node uncertainty for downstream applications that leverage landmark-level confidence.

    Source: §5 · p. 12

Abstract

from arXiv · show

In this work, we study uncertainty estimation for anatomical landmark-based segmentation on chest X-rays. Inspired by hybrid neural network architectures that combine standard image convolutional encoders with graph-based generative decoders, and leveraging their variational latent space, we derive two complementary measures: (i) latent uncertainty, captured directly from the learned distribution parameters, and (ii) predictive uncertainty, obtained by generating multiple stochastic output predictions from latent samples. Through controlled corruption experiments we show that both uncertainty measures increase with perturbation severity, reflecting both global and local degradation. We demonstrate that these uncertainty signals can identify unreliable predictions by comparing with manual ground-truth, and support out-of-distribution detection on the CheXmask dataset. More importantly, we release CheXmask-U (huggingface.co/datasets/mcosarinsky/CheXmask-U), a large scale dataset of 657,566 chest X-ray landmark segmentations with per-node uncertainty estimates, enabling researchers to account for spatial variations in segmentation quality when using these anatomical masks. Our findings establish uncertainty estimation as a promising direction to enhance robustness and safe deployment of landmark-based anatomical segmentation methods in chest X-ray. A fully working interactive demo of the method is available at huggingface.co/spaces/matiasky/CheXmask-U and the source code at github.com/mcosarinsky/CheXmask-U.

1. Introduction

The introduction identifies a gap in uncertainty estimation for topology-preserving landmark segmentation and presents a variational framework, validation experiments, and a large per-node uncertainty dataset. It positions fine-grained confidence as useful for assessing regional segmentation reliability.

1. Introduction

  • Quantifying confidence can help clinicians identify potentially unreliable segmentation predictions and support human intervention.The motivation connects this to improving diagnostic reliability.Source: §1 · p. 1
  • Pixel-based approaches can produce anatomically implausible predictions, while uncertainty estimation for topology-preserving landmark models had not been addressed.The introduction frames this as a gap in the practical use of hybrid landmark-segmentation architectures.Source: §1 · p. 2
  • The framework uses HybridGNet’s variational structure to quantify uncertainty in landmark-based segmentation through latent-space and predictive measures.The introduction describes lung and heart segmentation in chest X-rays, with predictive estimates generated through stochastic landmark predictions.Source: §1 · pp. 2–3
  • Controlled corruption experiments validate the uncertainty measures through correlation with landmark errors and out-of-distribution detection.The framework captures both latent-space and predictive uncertainty in a variational CNN–graph model.Source: §1 · p. 3
  • The authors release CheXmask-U, a dataset of 657,566 chest X-ray landmark segmentations with precomputed per-node uncertainty estimates.The dataset supports spatially informed use of anatomical masks, unlike image-level quality assessments alone.Source: §1 · pp. 2–3

2. Related work

Related work contrasts structured landmark representations with dense pixel masks, then reviews uncertainty estimation methods that largely remain pixel-based. It identifies landmark-level uncertainty as an underexplored complement for revealing confidence in specific anatomical points.

2.1. Landmark-based segmentation.

  • Landmark-based methods encode anatomical relationships and topology, whereas earlier statistical shape models relied on hand-crafted features and had limited flexibility for anatomical variability.HybridGNet combines CNNs and GCNNs to decode landmarks while preserving topology and spatial coherence.Source: §2.1 · p. 3
  • Existing landmark frameworks treat predictions deterministically, leaving landmark-position uncertainty unaccounted for in settings with occlusion, distortion, artifacts, or pathology.The text presents uncertainty as a way to identify unreliable landmarks and support selective trust in downstream decisions.Source: §2.1 · p. 3

2.2. Uncertainty estimation.

  • Uncertainty estimation distinguishes aleatoric uncertainty from data noise or annotation ambiguity and epistemic uncertainty from model limitations or insufficient training data.The paper focuses on latent-variable sampling without marginalizing over model parameters.Source: §2.2 · pp. 3–4
  • Pixel-level uncertainty methods include Bayesian inference, Monte Carlo dropout, ensembles, test-time augmentation, and entropy-based estimates; probabilistic networks model multiple segmentation hypotheses.The text notes computational costs for Bayesian inference and ensembles, and reports entropy estimates correlated with erroneous areas (Matzkin et al., 2025; Larrazabal et al., 2021).Source: §2.2 · p. 4
  • Landmark-level uncertainty remains largely unexplored despite landmark representations’ topological guarantees, and can reveal confidence in specific anatomical points.The text connects this fine-grained transparency to selective reliance in downstream clinical workflows.Source: §2.2 · p. 4

3. Method

HybridGNet encodes images into a probabilistic latent space and decodes graph-structured landmarks. The method estimates uncertainty both from latent distributions and from variability across sampled landmark predictions.

3.1. HybridGNet Architecture

  • HybridGNet represents an organ ROI as a graph whose nodes are landmarks and whose edges encode anatomical adjacency.Its CNN encoder extracts image features, while a GCNN decoder predicts landmark coordinates using the shared anatomical connectivity.Source: §3.1 · p. 4
  • The VAE latent representation is modeled as Q(z|I) = N(µ, σ²), and its variance provides latent uncertainty for an input image.Figure 1 depicts the image encoding into a probabilistic latent space and multiple latent samples.Source: §3.1 · p. 4, Figure 1 · p. 5

3.2. Uncertainty Estimation

  • Node-wise predictive uncertainty is estimated by sampling latent vectors, decoding multiple landmark graphs, and computing per-node variance.The resulting node-wise distributions capture predictive uncertainty, reflecting model uncertainty and data ambiguities.Source: §3.2 · p. 5, §3.1 · p. 5
  • Under occlusion, nodes inside masked regions show higher uncertainty than nodes in unaffected or visible regions.Figure 2 visualizes node-level uncertainty and compares values for nodes inside and outside occlusion blocks.Source: Figure 2 · p. 6
  • The method performs one image encoding per input and can batch the N stochastic decodings to generate multiple predictions efficiently.This allows multiple predictions without repeating image encoding.Source: §3.2 · p. 6

4. Experiments and Results

The experiments evaluate uncertainty under corruption, for reliability and OOD detection, and across a dataset release and pixel-level baselines. They report that uncertainty tracks degradation and that the variational HybridGNet provides a stronger global uncertainty–error signal than pixel-level baselines.

4.1. Dataset, implementation and training details

  • Occluded landmarks had significantly larger predictive uncertainty than visible landmarks, indicating localization of confidence degradation under partial information loss.The experiment used artificial black-square masks and 50 stochastic predictions.Source: §4.1 · p. 7
  • Under Gaussian noise, latent uncertainty increased and saturated for both variants, while predictive uncertainty declined at severe corruption for the skip-connections variant.The authors suggest that high-resolution features bypassing the variational bottleneck may reduce interpretability at large corruption levels.Source: §4.1 · p. 7
  • Predictive-uncertainty OOD detection achieved AUCs of 0.98 with skip-connections and 0.93 without; latent-space Isolation Forest scores achieved 0.93 and 0.89, respectively.Images with RCA-DSC below 0.7 were treated as OOD.Source: §4.1 · pp. 7, 9

4.2. The CheXmask-U Dataset

  • CheXmask-U adds per-node uncertainty estimates to 657,566 chest X-ray landmark segmentations, providing spatial reliability information beyond image-level RCA-Dice.The estimates are precomputed alongside landmark coordinates and standard deviations from 50 stochastic samples per image.Source: §4.2 · pp. 9–10
  • Across 255 manually annotated images, higher predicted node uncertainty corresponded to larger observed landmark errors for both annotators.The reliability analysis binned predictions by uncertainty quantile and compared mean predicted uncertainty with mean landmark error.Source: §4.2 · p. 10, Figure 5 · p. 10
  • Images with higher mean node-wise uncertainty tended to have lower RCA-estimated Dice scores, while individual nodes revealed region-specific uncertainty.The image-level comparison uses mean node-wise standard deviation as an overall uncertainty proxy.Source: §4.2 · pp. 10–11

4.3. Comparison with pixel-level UQ baselines

  • Variational HybridGNet showed stronger uncertainty–error correlation than pixel-level baselines, while node-level estimates retain spatial detail unavailable from pixel-level UQ.The comparison evaluates predictive entropy against segmentation error (1 - Dice).Source: Figure 6 · p. 11, §4.3 · p. 12
  • The proposed method encodes each image once and performs multiple stochastic latent decodings, reducing uncertainty-estimation overhead relative to methods requiring a forward pass per sample.Training used lung and heart landmarks from JSRT, Padchest, Montgomery, and Shenzhen; the KL weight was increased during training.Source: §4.1 · pp. 6–7, §4.3 · p. 12

5. Conclusion

The framework estimates node-level uncertainty for anatomical landmarks, and both uncertainty measures respond to perturbations and indicate out-of-distribution chest X-rays. CheXmask-U provides per-node estimates for 657,566 chest X-ray landmark segmentations.

5. Conclusion

  • The framework leverages HybridGNet’s VAE structure to estimate uncertainty in landmark-based anatomical segmentation.The authors characterize uncertainty estimation as feasible and valuable, with interpretability and safety beyond pixel-level approaches.Source: §5 · p. 12
  • Both latent-space and node-wise predictive uncertainties respond to image perturbations and indicate out-of-distribution chest X-rays.Comparisons with pixel-level uncertainty methods found the signals complementary, spatially informative, and lower in computational overhead.Source: §5 · p. 12
  • CheXmask-U releases 657,566 chest X-ray landmark segmentations with per-node uncertainty estimates for downstream use.The authors describe this as the first work to provide node-level uncertainty for anatomical landmarks.Source: §5 · p. 12
  • Future work could extend the framework to multi-organ or 3D imaging.Source: §5 · p. 12

Appendix A. Sensitivity to number of samples

Uncertainty–error agreement remains stable over a wide range of Monte Carlo sample counts and saturates before N = 50, suggesting moderate sampling is sufficient in practice.

Appendix A. Sensitivity to number of samples

  • A moderate number of Monte Carlo samples is sufficient in practice for the reported uncertainty estimates.Source: Appendix A · p. 16
  • Uncertainty–error agreement remains stable across sample counts and saturates well before N = 50.Table 1 reports sensitivity of uncertainty–error correlation to Monte Carlo sample count.Source: Table 1 · p. 16, Appendix A · p. 16
  • The analysis compares predicted-uncertainty and landmark-error correlation against inter-annotator discrepancies, and reports total runtime across 255 annotations.Source: Appendix A · p. 16
Loading 2512.10715v2…