Source-linked AI summary
Beyond the Pixel-Wise Loss for Topology-Aware Delineation
Agata Mosinska, Pablo Marquez-Neila, Mateusz Kozinski, Pascal Fua
TL;DR
Curvilinear delineation is poorly served by pixel-wise losses because they do not reflect topology-sensitive mistakes. The paper adds a topology-aware loss and shared-parameter iterative refinement, reporting improved delineations across microscopy and aerial-image tasks, including a leading single-classifier EM challenge score.
Problem
Pixel-wise losses such as binary cross-entropy treat curvilinear-structure errors locally and cannot reflect their topological impact.
Method
The paper combines a topology loss based on pretrained VGG19 feature maps with iterative refinement that reuses one model across steps.
Results
The approach outperforms baselines across three datasets, with larger differences on linear-structure and topology metrics, and achieves a 0.981 foreground-restricted random score on the EM challenge.
Takeaways & Limitations
Topology-aware training and parameter-sharing refinement improve delineation quality while keeping the model generic and applicable to roads, cracks, and neuronal membranes.
Takeaways & Limitations
The authors identify learning the topology loss with adversarial networks as future work because GAN training can suffer from mode collapse and instability.
Abstract
from arXiv · showhide
Delineation of curvilinear structures is an important problem in Computer Vision with multiple practical applications. With the advent of Deep Learning, many current approaches on automatic delineation have focused on finding more powerful deep architectures, but have continued using the habitual pixel-wise losses such as binary cross-entropy. In this paper we claim that pixel-wise losses alone are unsuitable for this problem because of their inability to reflect the topological impact of mistakes in the final prediction. We propose a new loss term that is aware of the higher-order topological features of linear structures. We also introduce a refinement pipeline that iteratively applies the same model over the previous delineation to refine the predictions at each step while keeping the number of parameters and the complexity of the model constant. When combined with the standard pixel-wise loss, both our new loss term and our iterative refinement boost the quality of the predicted delineations, in some cases almost doubling the accuracy as compared to the same classifier trained with the binary cross-entropy alone. We show that our approach outperforms state-of-the-art methods on a wide range of data, from microscopy to aerial images.
1. Introduction
Curvilinear-structure delineation remains difficult in noisy, complex images, while prevailing approaches rely on local pixel-wise losses that miss topology. The paper introduces a topology-aware loss and shared-parameter iterative refinement to improve predictions without changing the network architecture.
- Automated delineation remains elusive for noisy image data and complex curvilinear structures despite decades of research.
- Pixel-wise binary cross-entropy treats mistakes locally and equally, despite their potentially large effects on curvilinear topology.
- The proposed topology loss compares higher-level linear-structure descriptions from pretrained VGG19 feature maps for ground truth and predictions.
- Iterative refinement repeatedly applies the same architecture and parameters, keeping parameter count constant across refinement steps.This design is important when labeled training data is limited.
- Accounting for topology in the training loss is presented as an important step toward boosting delineation performance without changing the network architecture.
2. Related Work
Earlier delineation methods range from hand-crafted feature pipelines to learned classifiers and deep networks. Although these methods improve representation and context, standard cross-entropy still scores pixels independently, while refinement approaches can require costly stacks of modules.
- Hand-crafted OOF and MDOF methods avoid training data but struggle with irregular structures, scale variation, appearance variability, and artifacts.
- Learning-based methods use diverse features and prediction targets, including Haar wavelets, spectral features, and distance-to-centerline regression.
- Deep networks expanded receptive fields and supported road and biomedical delineation, including CNNs, differentiable IoU, VGG-based models, and U-Net.
- Standard cross-entropy treats pixels independently and ignores higher-level statistics, even with structured-output architectures such as U-Net.
- Higher-level topology has also been modeled using CRFs over connected superpixels, but this requires sampling and hand-designed features because of many potential cliques.
- Deep-learning refinement commonly stacks modules, increasing parameters and resource demands when time, memory, or training data are limited.
3. Method
The method combines a topology-aware loss with iterative U-Net refinement to improve curvilinear-structure delineations while preserving a constant model size across iterations.
- Topology-aware loss: The U-Net predicts curvilinear structures, while BCE scores pixels independently and misses topological properties such as connectivity and holes.A few pixel errors can have a large effect on thin-structure topology despite a low BCE cost.
- Topology-aware loss: The topology loss compares pretrained VGG19 feature responses of the ground-truth and predicted delineations.These higher-level features capture visual characteristics associated with connectivity, holes, and linear structures.
- Topology-aware loss: Ltop penalizes structurally disruptive errors and small non-linear false positives more strongly than pixel-wise BCE.VGG19 channels responding to elongated structures are encouraged, whereas channels responding to small connected components are penalized.
- Topology-aware loss: The training objective minimizes Lbce(x, y, w) + µLtop(x, y, w), balancing pixel-wise accuracy with topology-aware differences.The scalar µ is chosen so the two loss terms have comparable magnitudes.
- Iterative refinement: Iterative refinement repeatedly applies the same U-Net to the input image and previous prediction, producing successive delineation estimates without adding parameters per iteration.The refinement loss is a weighted sum of partial losses, with greater weight assigned to later iterations.
4. Results
The experiments evaluate the method on three datasets using pixel-wise, linear-structure, and topology-based measures, with results favoring topology-aware loss and iterative refinement. Qualitative examples show fewer background false positives, preserved structures, and closed gaps.
- Evaluation setup: The approach is evaluated on cracks, roads, and neuronal structures using three datasets with substantially different linear-structure characteristics.The datasets include asphalt crack images, aerial road images, and electron-microscopy imagery.
- Evaluation setup: Correctness, completeness, and quality measure similarity between predicted and ground-truth skeletons while tolerating limited centerline displacement.These metrics emphasize centerline alignment over exact structure width and relax true-positive matching by a distance threshold.
- Evaluation setup: Topology-based metrics sample connected paths and classify predictions as correct, infeasible, too-long, or too-short according to their correspondence with ground-truth paths.The EM challenge’s foreground-restricted random score instead evaluates whether pixels belonging to the same cell remain connected in the prediction.
- Configuration: The best configuration uses the first three VGG layers for topology loss, while performance stabilizes after the third refinement iteration.The selected configuration therefore uses three refinement iterations in subsequent experiments.
- Quantitative results: Across the three datasets, topology loss outperforms all baselines without refinement, and refinement improves performance further, especially on linear-structure and topology-based metrics.Per-pixel improvements suggest that better topology is correlated with better localization.
- Quantitative results: The EM challenge score is 0.981, placing the method first among single-classifier algorithms; the second-ranked method scores 0.978 with a more complex base classifier.The score is a foreground-restricted random score.
- Qualitative results: Qualitative predictions become more structured, remove background false positives, preserve target structures, and close small gaps during refinement.Examples include filtering roofs and rivers from road images, preserving and enhancing roads, and removing mitochondria-like false positives from neuronal membranes.
5. Conclusion
The paper introduces a topology-aware loss and a recursive refinement stage for curvilinear-structure delineation. The approach is presented as generic across linear-structure types and network architectures, with future work targeting more discriminative topological features.
- The topology-aware loss accounts for higher-level features of curvilinear structures, while recursive refinement improves predictions without increasing learned parameters.
- The approach is demonstrated with U-Net but can be used with other network architectures.
- The method is intended for roads and cracks in natural images and neuronal membranes in micrographs.
- Future work will explore adversarial networks to adapt topological similarity measures and learn more discriminative features.