Source-linked AI summary
blob loss: instance imbalance aware loss functions for semantic segmentation
Florian Kofler, Suprosanna Shit, Ivan Ezhov, Lucas Fidon, Izabela Horvath, Rami Al-Maskari, Hongwei Li, Harsharan Bhatia, Timo Loehr, Marie Piraud, Ali Erturk, Jan Kirschke, Jan C. Peeken, Tom Vercauteren, Claus Zimmer, Benedikt Wiestler, Bjoern Menze
TL;DR
Volumetric losses such as Dice do not account for instance imbalance, allowing large foreground objects to dominate while small clinically important instances are missed. Blob loss converts existing semantic-segmentation losses into instance-aware objectives by computing loss terms for individual connected components and combining them with a global loss. Across five complex 3D biomedical segmentation tasks, extending soft Dice with blob loss significantly improves detection performance and sometimes volumetric performance.
Problem
Volumetric losses such as Dice do not account for instance imbalance, allowing large foreground objects to dominate while small clinically important instances are missed.
Method
Blob loss converts existing semantic-segmentation losses into instance-aware objectives by computing loss terms for individual connected components and combining them with a global loss.
Results
Across five complex 3D biomedical segmentation tasks, extending soft Dice with blob loss significantly improves detection performance and sometimes volumetric performance.
Takeaways & Limitations
Blob loss provides a practical way to add instance-imbalance awareness to semantic segmentation when detecting multiple small structures matters.
Takeaways & Limitations
Blob loss depends on instance segmentation labels or connected-component preprocessing and requires larger patches, increasing computational and GPU-memory demands, especially for 3D data.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNN) have proven to be remarkably effective in semantic segmentation tasks. Most popular loss functions were introduced targeting improved volumetric scores, such as the Dice coefficient (DSC). By design, DSC can tackle class imbalance, however, it does not recognize instance imbalance within a class. As a result, a large foreground instance can dominate minor instances and still produce a satisfactory DSC. Nevertheless, detecting tiny instances is crucial for many applications, such as disease monitoring. For example, it is imperative to locate and surveil small-scale lesions in the follow-up of multiple sclerosis patients. We propose a novel family of loss functions, \emph{blob loss}, primarily aimed at maximizing instance-level detection metrics, such as F1 score and sensitivity. \emph{Blob loss} is designed for semantic segmentation problems where detecting multiple instances matters. We extensively evaluate a DSC-based \emph{blob loss} in five complex 3D semantic segmentation tasks featuring pronounced instance heterogeneity in terms of texture and morphology. Compared to soft Dice loss, we achieve 5% improvement for MS lesions, 3% improvement for liver tumor, and an average 2% improvement for microscopy segmentation tasks considering F1 score.
1 Introduction
Semantic segmentation losses can achieve strong volumetric scores while overlooking instance imbalance, allowing large objects to dominate smaller ones. Blob loss addresses this gap by adding instance awareness without requiring instance-wise prediction.
- Problem: Large foreground instances can dominate volumetric losses, causing small but clinically important objects to be missed despite satisfactory segmentation scores.This issue is especially relevant in medical applications such as multiple sclerosis, where detecting one lesion can affect treatment decisions.
- Problem: Existing approaches compensate for class imbalance, but a notable gap remains in handling imbalance among instances within semantic segmentation.Instance differences can involve size, texture, and morphology, while established metrics may correlate insufficiently with expert assessment.
- Contribution: Blob loss converts an existing loss into an instance-imbalance-aware objective by assigning a specific loss term to each instance without requiring instance-wise predictions.The framework is designed for semantic segmentation tasks where detection of multiple instances matters.
- Contribution: Across five complex 3D biomedical segmentation tasks, extending soft Dice with blob loss improves detection performance and sometimes volumetric performance.The evaluated tasks include medical imaging and microscopy problems involving miniature structures.
2 Methods
Blob loss isolates each foreground instance through masking, applies the base loss locally, and averages the resulting instance terms to reduce size-driven dominance. A global loss component is combined with this instance-aware component for training standard 3D U-Nets.
- Problem statement: Large foreground areas dominate established volumetric metrics and losses, motivating an instance-aware formulation.Voxel-level accumulation does not represent whether individual instances are detected, so volumetry-based training can yield poor instance detection.
- Blob loss construction: For each instance, blob loss excludes the other foreground objects from the image domain while retaining the background in the instance-specific domain.The resulting masks are constructed from connected foreground components and are applied to the network outputs.
- Blob loss construction: The method converts any binary semantic-segmentation loss into an instance-aware loss by computing the base loss separately for each masked instance.The instance-specific losses are averaged so instances receive equal importance regardless of size, shape, texture, or other topological attributes.
- Total loss: The final training objective combines a global loss component with the instance-wise blob component using weights α and β.The global term preserves whole-volume supervision while the blob term adds the instance constraint.
- Experimental setup: Experiments use a basic 3D U-Net with soft Dice and Tversky as comparison losses, random foreground-centered crops, and connected-component preprocessing.The study avoids a separate validation split for most datasets because expert annotations are limited, instead using an 80:20 train-test split and the last checkpoint.
- Evaluation: Evaluation includes volumetric DSC, sensitivity, precision, and surface Dice, together with instance F1, sensitivity, and precision.These metrics jointly assess voxel-level segmentation and instance-level detection.
3 Experiments
The experiments test blob loss across heterogeneous 3D biomedical segmentation tasks spanning brain MRI, thorax CT, and light-sheet microscopy. The datasets contain fragmented structures and vary in blob counts, annotation quality, and imaging characteristics.
- Datasets: Blob loss is evaluated across brain MRI, thorax CT, and light-sheet microscopy datasets containing fragmented semantic-segmentation problems.The study uses multiple modalities to assess performance across heterogeneous biomedical structures.
- Liver tumor: The liver-tumor experiments use 500 epochs, batch size 2, two GPUs, 192x192x64 crops, and Hounsfield-unit windowing tailored to liver tumors.The normalization window is centered around 30 HU with width 150 HU and added tolerance.
- Light-sheet microscopy: DISCO-MS contains 41 training and six testing volumes with 988 annotated blobs, while DeepMACT contains 115 training and 19 testing images with 484 annotated blobs.The microscopy datasets differ in blob density and annotation sparsity, requiring careful interpretation of results.
- Annotation characteristics: The microscopy annotations and connected-component overlays reveal missed objects and imperfect contours, reflecting ambiguity in the light-sheet signal and limitations also seen in MS and LiTS annotations.These annotation characteristics qualify interpretation of segmentation performance across the experiments.
4 Results
Across five datasets, blob loss improves instance-detection metrics over conventional losses and sometimes improves volumetric performance, while remaining robust to hyperparameter choices when the global term is included.
- Overall results: Blob loss improves detection metrics across all five datasets and sometimes also improves volumetric measures.The study compares blob loss with Dice, Tversky, and inverse weighting; inverse weighting produced over-segmentations despite mitigation strategies.
- Hyperparameter ablation: α = 2 and β = 1 provide the best results in the MS-lesion hyperparameter ablation.The ablation indicates that higher importance assigned to the global parameter performs best.
- Hyperparameter ablation: Blob loss remains robust to hyperparameter choice as long as the global term is included with α greater than 0.
5 Discussion
Blob loss extends existing losses with instance imbalance awareness and improves detection across diverse 3D biomedical segmentation tasks, but requires instance labels and additional computation.
- Contribution: Blob loss adds instance imbalance awareness to existing losses and improves detection, with occasional gains in volumetric segmentation, across five complex 3D biomedical tasks.Evaluated applications include MS lesions, liver tumors, and biological structures in light-sheet microscopy.
- Limitations: Blob loss requires instance segmentation labels, although connected-component analysis can often provide them.
- Limitations: Blob loss requires larger patches containing multiple instances, increasing GPU-memory demands, especially for 3D data.
- Interpretation: The authors hypothesize that per-instance attention may encourage learning morphology and texture differences, while multiple instance terms may improve precision by penalizing background.The proposed explanation is explicitly speculative.
- Outlook: Future work must determine how broadly blob loss transfers to other segmentation tasks and loss functions.
6 Supplementary Materials
The supplementary materials define evaluation abbreviations and dataset-shape descriptors, document data-access constraints, and show that masking is crucial for segmentation performance.
- Definitions: The supplementary materials define DSC, SDSC, instance sensitivity, and instance precision as evaluation abbreviations.
- Data Set Description: Figure 42 summarizes dataset heterogeneity using blob counts, blob volumes, and distributions of compactness, sphereness, stringness, and skewness.
- Data access: MS data requests require ethical review and supporting research, ethics, and data-transfer documentation, while other datasets have differing access routes.
- Why do we need masking?: Masking is crucial for high segmentation performance; removing it causes over-segmentation, with high instance sensitivity but low instance precision and degraded DSC and SDSC.
6.5 Qualitative Segmentation Performance
Qualitative examples show blob loss improving lesion detection and volumetric Dice relative to Dice loss, with detection gains driven by fewer missed lesions and false positives.
- Detection performance: F1 rises from 0.74 to 1.0 when Dice loss is transformed into blob loss for the illustrated patient.Instance sensitivity increases from 0.83 to 1.0 and instance precision from 0.67 to 1.0.
- Volumetric segmentation performance: Volumetric Dice increases from 0.56 to 0.70 for the illustrated patient after applying blob loss.
6.6 MS model selection experiment
The MS model-selection experiment evaluates checkpoints using validation loss, revealing poorer generalization for conventional losses and more robust performance with blob loss.
- Blob loss training leads to more robust performance than conventional losses in MS model selection.The experiment evaluates checkpoints using the lowest validation loss on 21 patients; Dice and especially Tversky show large performance drops.
6.7 Multi-class Segmentation Extension of blob loss
Blob loss extends from binary to multi-class semantic segmentation by summing instance-aware losses across foreground classes, with optional class-specific weighting.
- Each foreground class uses an instance-aware domain that excludes voxels belonging to other instances of that class.The domain Ωc,n excludes voxels labeled as class c that do not belong to instance n.
- The multi-class extension sums the binary blob loss across foreground classes.Background is assigned class 0, while C ≥ 1 denotes the number of foreground classes.
- The multi-class loss permits nonnegative α and β weights, which may be made class-specific for a given task.The formulation retains α ≥ 0 and β ≥ 0 and allows separate values across foreground classes.
6.8 Dependency on Network architecture
The architecture study tests whether blob-loss improvements depend on network design, comparing standard U-Net and transformer-based UNETR models on MS data.
- 6.8 Dependency on Network architecture: The primary experiments use a standard U-Net because the study focuses on evaluating the loss function rather than optimizing architecture.The U-Net has encoder and decoder parts and is implemented through MONAI, inspired by Falk et al. [8].
- 6.8 Dependency on Network architecture: The MS architecture ablation keeps the learning rate at 1e-3 and other training parameters constant across runs.The comparison includes blob Dice and Dice baselines with UNETR.
- 6.8 Dependency on Network architecture: Blob-loss performance improvements are architecture agnostic in the UNETR ablation study.The study compares Dice with blob Dice while training UNETR models, whose transformer components differ from the basic U-Net.
6.9 Increased Instance-wise Penalization of False Positives for Soft Dice Loss
Blob loss increases false-positive penalization by applying instance-specific denominator terms, with the penalty growing as the number of instances increases.
- Although all instances are predicted in one channel, the prediction can be decomposed into true-positive and false-positive components for analysis.The construction uses instance masks whose union equals the binary foreground segmentation.
- The instance-aware soft Dice loss penalizes false positives more strongly than soft Dice loss.False-positive terms appear in the denominator for every instance, creating a dynamic penalty.
- The false-positive penalty is proportional to the number of instances, limiting over-prediction when many instances are present.The penalty has little effect when fewer instances are present.