Source-linked AI summary
Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations
Reuben R Shamir, Yuval Duchin, Jinyoung Kim, Guillermo Sapiro, Noam Harel
TL;DR
Classical binary overlap measures are poorly suited to probabilistic segmentations and are biased by structure size. The paper introduces a closed-form continuous Dice coefficient that directly compares probabilistic maps with binary ground truth without thresholding. Across simulations and automatic subthalamic-nucleus segmentation, cDC was less biased and more robust to partial-volume effects than classical Dice.
Problem
Classical Dice-based overlap measures require binary inputs, while modern segmentation methods produce probabilistic maps and Dice values vary with structure size.
Method
The paper extends Dice with a closed-form continuous Dice coefficient that directly compares probabilistic segmentations with binary ground truth without thresholding.
Results
The cDC was less biased and more robust to partial-volume effects and structure size than classical Dice, including STN values of 0.66 for DC and 0.80 for cDC.
Takeaways & Limitations
cDC provides a similarity measure for probabilistic segmentation that weights errors according to confidence or probability.
Abstract
from arXiv · showhide
Objective: Overlapping measures are often utilized to quantify the similarity between two binary regions. However, modern segmentation algorithms output a probability or confidence map with continuous values in the zero-to-one interval. Moreover, these binary overlapping measures are biased to structure size. Addressing these challenges is the objective of this work. Methods: We extend the definition of the classical Dice coefficient (DC) overlap to facilitate the direct comparison of a ground truth binary image with a probabilistic map. We call the extended method continuous Dice coefficient (cDC) and show that 1) cDC is less or equal to 1 and cDC = 1 if-and-only-if the structures overlap is complete, and, 2) cDC is monotonically decreasing with the amount of overlap. We compare the classical DC and the cDC in a simulation of partial volume effects that incorporates segmentations of common targets for deep-brainstimulation. Lastly, we investigate the cDC for an automatic segmentation of the subthalamic-nucleus. Results: Partial volume effect simulation on thalamus (large structure) resulted with DC and cDC averages (SD) of 0.98 (0.006) and 0.99 (0.001), respectively. For subthalamic-nucleus (small structure) DC and cDC were 0.86 (0.025) and 0.97 (0.006), respectively. The DC and cDC for automatic STN segmentation were 0.66 and 0.80, respectively. Conclusion: The cDC is well defined for probabilistic segmentation, less biased to structure size and more robust to partial volume effects in comparison to DC. Significance: The proposed method facilitates a better evaluation of segmentation algorithms. As a better measurement tool, it opens the door for the development of better segmentation methods.
I. INTRODUCTION
Medical-image segmentation supports surgical planning and disease monitoring, but classical overlap evaluation is limited when methods produce probabilistic maps and when structure sizes differ.
- Accurate anatomical or pathological segmentation facilitates safer surgical planning and quantitative monitoring of disease progression.
- The Dice coefficient compares computed and ground-truth segmented regions but operates on binary data.
- Smaller structures receive lower Dice coefficients at fixed resolution, complicating comparisons across structures and the design of segmentation methods.
- Probabilistic maps contain values in [0, 1], yet common overlap measures require thresholding them into binary images, altering the original segmentation.
- The proposed continuous Dice coefficient is a closed-form extension that avoids thresholding while addressing structure-size dependence and probabilistic segmentation.
II. CONTINUOUS DICE COEFFICIENT
The continuous Dice coefficient (cDC) extends Dice to compare binary ground truth with probabilistic segmentations directly, without thresholding. It preserves the classical Dice value for binary maps while satisfying boundedness and overlap-related properties.
- The classical Dice coefficient is defined for binary masks, whereas probabilistic maps assign real values in [0, 1].
- Thresholding a probabilistic map changes the original segmentation and discards confidence information, especially at boundaries affected by resolution and partial volume effects.
- The cDC extends Dice by weighting the ground-truth size so complete overlap receives a score of 1 without thresholding.
- For binary-valued B, cDC equals the classical DC, making it a consistent extension to real-valued probabilistic maps.
- The formulation assumes a binary ground-truth mask, although it can be extended to probabilistic ground truth.
- The cDC is at most 1 and equals 1 if and only if the ground-truth and computed structures completely overlap.
function = continuous_dice_coefficient (A,B)
The implementation computes the intersection size and the individual sizes of the two input images using elementwise products and summation.
- size_of_A_intersect_B is computed by summing the elementwise product of A and B.
- size_of_A is computed by summing all elements of A.
- size_of_B is computed by summing all elements of B.
if (size_of_A_intersect_B > 0)
The cDC is empirically evaluated by shifting a simulated probabilistic STN map to model segmentation errors and partial volume effects.
- A probabilistic STN map was simulated with a Gaussian distribution over a manually segmented image.
- A 2mm translation shifted the probabilistic map to simulate a simple segmentation error.
- A random-direction half-voxel translation simulated partial volume effects for comparing cDC with DC.
III. COMPARISON WITH DICE COEFFICIENT
The study evaluates cDC against classical DC using simulated probabilistic segmentations and an automatic STN segmentation, with DC computed after thresholding.
- cDC implementation and empirical evaluation used MATLAB, with its properties tested on manipulated clinical data.
- Manual STN, GP, and thalamus segmentations were converted into Gaussian probabilistic maps and translated by 0.25mm to simulate partial volume effects.
- An automatic right-STN probabilistic map was generated from standard clinical MRI using similar registered clinical and 7T MRI cases.
- The DC comparison used a binary image created by applying a 0.1 threshold, whereas cDC was computed directly from the probabilistic map.
IV. RESULTS
Simulations confirmed the proposed properties and showed that cDC was less affected by partial volume effects than DC, while automatic STN evaluation also favored cDC.
- DC 0.86 (0.025) and cDC 0.97 (0.006) were the STN averages (SD) under simulated partial volume effects.
- The simulations showed that cDC was less biased and more robust to partial volume effects than classical DC.
- DC 0.66 and cDC 0.80 were obtained for automatically segmented versus ground-truth right STNs.
V. DISCUSSION
The discussion concludes that cDC evaluates probabilistic segmentations directly while accounting for partial errors, low-confidence voxels, partial volume effects, and structure size.
- cDC directly measures similarity between a binary ground truth and a probabilistic segmented image without thresholding.
- cDC accounts for partial errors and low-confidence voxels that classical thresholded overlap measures treat equally.
- For small structures, cDC reduces size-related bias and produced 0.8 versus DC 0.66 in the STN experiment.
VI. CONCLUSION
The continuous Dice coefficient directly compares probabilistic maps with ground-truth segmentations and weights errors by confidence or probability. It is reported as less biased and more robust than classical Dice coefficient.
- cDC directly compares a computed probabilistic map with ground-truth segmentations.
- cDC weights segmentation errors according to confidence or probability, whereas classical Dice coefficient rates them all the same.
- The authors report that cDC satisfies desired properties and is less biased and more robust than classical Dice coefficient.
- The proposed measure is intended to support studies of probabilistic segmentation methods and the design and analysis of new techniques.