Source-linked AI summary
Deep Vessel Segmentation By Learning Graphical Connectivity
Seung Yeon Shin, Soochahn Lee, Il Dong Yun, Kyoung Mu Lee
TL;DR
CNN-based vessel segmentation has largely focused on local image-grid appearances rather than vessel-shape connectivity. The paper introduces VGN, which combines CNN and GCN features to infer segmentation, and reports improved performance across retinal and coronary angiography datasets.
Problem
CNN-based vessel segmentation methods have mostly learned local appearances on regular image grids without modeling the graphical structure of vessel shape.
Method
VGN jointly trains a CNN for pixelwise features and probabilities with a GCN operating on a graph constructed from vessel centerlines, then combines their features for final segmentation.
Results
VGN outperforms current state-of-the-art methods on two retinal datasets and a coronary artery X-ray angiography dataset.
Takeaways & Limitations
The architecture is broadly applicable because it can be combined with any CNN-based vessel segmentation method.
Abstract
from arXiv · showhide
We propose a novel deep-learning-based system for vessel segmentation. Existing methods using CNNs have mostly relied on local appearances learned on the regular image grid, without considering the graphical structure of vessel shape. To address this, we incorporate a graph convolutional network into a unified CNN architecture, where the final segmentation is inferred by combining the different types of features. The proposed method can be applied to expand any type of CNN-based vessel segmentation method to enhance the performance. Experiments show that the proposed method outperforms the current state-of-the-art methods on two retinal image datasets as well as a coronary artery X-ray angiography dataset.
1 Introduction
The VGN addresses the limited modeling of vessel connectivity in CNN-based segmentation by jointly learning local appearances and global vessel structure. It combines CNN and GCN components in an architecture intended to extend CNN-based methods and improve segmentation performance.
- Manual vessel inspection can be inaccurate and time-consuming, motivating automatic vessel segmentation methods.
- Existing learning-based methods mostly learn local appearances, while optimization methods often rely on simple prior rules that limit modeling capacity.
- The vessel graph network jointly learns global vessel-shape structure and local appearances within a CNN architecture.
- VGN uses a CNN for pixelwise features and vessel probabilities, a GCN for neighboring-vertex connectivity features, and an inference module for final segmentation.
- The graph construction module generates the GCN input graph, while the architecture is designed to combine with any CNN-based segmentation method.
- Evaluations on two retinal datasets and a coronary X-ray angiography dataset report that VGN outperforms current state-of-the-art methods.
2 Methods
VGN combines CNN-derived local appearance features with graph-based vessel connectivity features, then uses an inference module to produce the final segmentation. Graphs are constructed from CNN predictions, and the network is trained sequentially before joint end-to-end fine-tuning.
- Network architecture: The CNN learns pixelwise features and vessel probabilities on the regular image grid, while the GCN models vertices on an irregular vessel graph.The graph is constructed from points sampled from vessel centerlines of an initial CNN segmentation.
- Graph construction: The graph construction module thresholds and skeletonizes the initial vessel map, samples vertices along skeletons, and connects them using skeletal connectivity or geodesic distances.Vertices include equidistant skeleton samples, junctions, and endpoints.
- Graph convolutional network: The two-layer GCN propagates neighboring-vertex information using the adjacency matrix and outputs vessel probabilities for graph vertices.Its final vertex probabilities are obtained by applying a sigmoid to the final graph features.
- Inference module: GCN hidden features are reprojected from sparse graph vertices onto the pixel grid and concatenated with CNN features before convolutional inference produces the final vessel map.The combined tensor has dimensions h × w × (C_CNN + C_GCN), with zero-padding at non-vertex pixels.
- Network training: Training first pretrains the CNN, then jointly fine-tunes the CNN and the rest of VGN using the constructed graphs and a combined loss.During testing, feature generation, graph construction, GCN processing, and final inference are performed sequentially for each image.
3 Experimental Results
Experiments evaluate VGN on two retinal datasets and a coronary angiography dataset using quantitative and qualitative comparisons. VGN achieves the strongest reported AP results and improves over its DRIU baseline, while qualitative results show fewer false positives and false negatives.
- Datasets: The evaluation covers DRIVE, STARE, and CA-XRA, with DRIVE and STARE containing 40 and 20 images and CA-XRA containing 3,137 frames from 85 sequences.CA-XRA frames were treated as independent 512 × 512, 8-bit images.
- Experimental setup: VGN uses an independently implemented DRIU CNN baseline because the original training code was unavailable, with a modified CNN for wider vessel-width variation in CA-XRA.The retinal CNN uses C_CNN=64, while CA-XRA uses C_CNN=80.
- Quantitative evaluation: The comparison reports precision-recall curves, average precision, and maximum F1 scores against state-of-the-art and conventional methods.Multiple vessel-probability thresholds generate the precision-recall curves.
- Quantitative evaluation: VGN is comparable to original DRIU on DRIVE, performs best on STARE, and achieves the highest AP scores on both retinal datasets.The plotted comparisons include the second annotator as human performance and DRIU* as the authors’ implementation baseline.
- Quantitative evaluation: VGN reaches AP 0.915 versus DRIU 0.899 on CA-XRA, a relative improvement of 1.78%.The comparison is reported for the coronary artery X-ray angiography dataset.
- Qualitative evaluation: Qualitatively, VGN reduces false positives and false negatives relative to, including rib artifacts in CA-XRA, while weak STARE vessels may be suppressed.The authors also report better apparent performance on higher-quality images such as STARE, where vessel graphs are clearer.
4 Conclusion
The paper concludes that VGN learns graphical vessel structure alongside local appearance for segmentation. Experiments support its effectiveness across three datasets spanning two target organs, with future work targeting 3D and temporal imaging.
- VGN explicitly learns graphical vessel structure together with local appearance for vessel segmentation.
- Experiments demonstrate effectiveness across three datasets involving two different target organs.
- Future work will extend VGN to 3D imaging modalities and temporal information in video data such as fluoroscopic X-ray sequences.