Source-linked AI summary
Dual-Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms
Olatunji Omisore, Ahmed Elazab, Ali Shahidinejad, Fariza Sabrina
TL;DR
Simultaneous multiclass segmentation of catheterization angiograms remains challenging because images contain degraded, overlapping structures and severe class imbalance, while prior work largely focuses on binary tasks. The paper proposes a dual-part MLBNet with laterally branched encoders, class-specific decoder heads, and class-aware losses. Across evaluated datasets, the models separated background and foreground structures, although foreground performance was weaker in some phantom and animal sets and broader clinical tool coverage remains needed.
Problem
Catheterization angiograms are difficult to segment because of noise, low contrast, structural overlap, and severe class imbalance, while prior studies largely focus on binary or single-tool segmentation.
Method
The dual-part MLBNet uses laterally branched encoder blocks, class-specific decoder branches, and a class-aware objective for simultaneous background–vessel–tool segmentation.
Results
The models accurately delineated three-class structures across test sets, with macro F1 of 96.94% for Dataset 1, 97.28% for Dataset 2, 66.32% for the CathAction phantom set, and 77.29% for the animal set.
Takeaways & Limitations
The dual-part MLBNet can distinguish dominant background pixels from foreground structures while also localizing foreground classes, though foreground performance varies by task and dataset.
Takeaways & Limitations
Further runs are needed to examine average performance, improve decoder-branch consistency and generalization, include tools such as stents and balloons, and improve explainability.
Abstract
from arXiv · showhide
Catheterisation image processing requires segmentation models that are fast, accurate and explainable. While most of the existing studies usually focus on binary segmentation, there is a recent demand for simultaneous segmentation of multiple structures found in catheterization scenes. In this study, a dual-part MLBNet architecture is designed with multi-lateral encoder blocks and multi-head decoder branches for class-aware segmentation in cardiovascular catheterization scenes. Lateral branches in the encoder enables repeated feature extraction to learn diverse shared representations, while multiple decoder heads are used to introduce class-skewed branches that specialize in different structural properties in catheterization scenes. To analyze the performances of the dual-part MLBNet architecture, several multi-class segmentation angiogram data obtained during cardiovascular catheterization in phantom models, synthetic human-simulated aorta, and animal model are used for model training and evaluation. Results obtained showed the dual-part models could effectively separate guidewire, catheter, vessels and background pixels to their classes of memberships with high probability. The results demonstrate that all models were able to distinguish the dominant background class from foreground structures with high overall accuracy.
1. Introduction
Cardiovascular catheterization angiograms are difficult to segment because they contain low contrast, noise, motion artifacts, overlapping structures, and severe class imbalance. Existing deep-learning studies largely address binary or single-tool segmentation, motivating a dual-part MLBNet for simultaneous background, vessel, and tool segmentation.
- Angiographic images combine low contrast, noise, motion artifacts, cluttered class overlaps, and severe class imbalance.
- Classical approaches use thresholding, intensity analysis, clustering, preprocessing, and multi-scale filtering, but manual target definition is unsuitable for long or moving structures.
- Deep-learning studies have improved endovascular segmentation but primarily target binary segmentation or extraction of a single guidewire or catheter.
- Simultaneous segmentation of multiple catheterization-scene structures remains underexplored, with prior multiclass efforts omitting either blood vessels or comprehensive multi-structure segmentation.
- The proposed dual-part MLBNet uses lateral branching in encoder and decoder subnetworks for simultaneous background–vessel–tool pixel-level segmentation.It also introduces a hybrid objective with structure-specific loss terms to address class dominance and imbalance.
2. Multi-Class Segmentation Technique
The paper formulates catheterization angiogram segmentation as dense prediction of procedure-specific structures and proposes a dual-part MLBNet architecture. Its design combines laterally branched encoding with a decoder structure intended for simultaneous multiclass segmentation.
- The task maps an angiogram to a dense class-probability map for K procedure-specific structures against pixel-level one-hot labels.
- The proposed hypothesis is that lateral branching in encoding and decoding can support improved and stable simultaneous multiclass segmentation.
- The dual-part MLBNet design is presented as an encoder–decoder architecture with lateral branching.
- The encoder begins with a shallow stem block that extracts low-level spatial features before passing them through consecutive laterally branched segments.
A. Lateral Encoder Branching
The lateral encoder uses parallel convolutional paths to learn diverse representations from angiogram features. Depthwise, separable, pooling, residual, and atrous operations support thin-structure responses, feature diversity, dimensionality reduction, and multiscale context.
- Each encoder lateral branch contains three parallel convolutional paths that produce complementary feature maps from input images.
- Depthwise convolutions preserve local edge and line responses important for thin elongated structures.
- The middle branch uses separable convolution for feature diversity, while max-pooling reduces feature dimensionality.
- Branch outputs are combined through channel concatenation followed by projection or additive residual fusion when dimensions align.
- Atrous separable convolutions with adaptive dilation rates provide contextual multiscale features while reducing network parameters.
B. Multi-flow Structure
The network aggregates multiscale encoder features and decodes them through class-specific lateral branches. Its class-aware objective combines cross-entropy, focal, connectivity-sensitive, and Tversky components to address coupled structures and class imbalance.
- Multi-flow Structure: Encoder blocks are organized into entry, middle, and exit flows that capture low-level, intermediate, and contextual feature representations.Dilation and field-of-view expansion maintain consistent output dimensions across branches.
- Multi-flow Structure: Skip features from B3, B5, and B7 are aggregated and bilinearly upsampled to support multiscale deep supervision in the decoder.
- Decoder Sub-network: Because tool pixels often lie within or along continuous vessel structures, the decoder uses class-specific lateral branches rather than a single shared or sequential head.
- Decoder Sub-network: Separate decoder branches target tool, vessel, and background classes, then fuse branch outputs into a global class space.
- Class-Aware Scene Objective Function: The class-aware scene objective treats each branch as a one-vs-rest classifier with structured non-target supervision and adaptive weighting for competing classes.
- Class-Aware Scene Objective Function: A branch-wise Tversky loss improves overlap under class imbalance by controlling the penalty trade-off between false negatives and false positives.
A. Tool Branch Loss Term
The tool branch combines tool-focused supervision with structural constraints tailored to guidewires and catheters. It promotes thin, connected tool predictions contained within vessel support while distinguishing vessel pixels from background.
- A. Tool Branch Loss Term: The tool branch uses tool Dice and skeleton-consistency functions to supervise guidewire and catheter predictions.Skeleton consistency accounts for structure thinness, connectivity, and centerline.
- A. Tool Branch Loss Term: Structural containment penalizes tool predictions that fall outside the vessel support.
- A. Tool Branch Loss Term: Background receives stronger negative supervision than vessel because vessel pixels are anatomically related to the tool.
B. Vessel Branch Loss Term
The vessel branch uses combined overlap and boundary-aware supervision to model coherent vessel anatomy, tool boundaries, and support continuity. Its constraints also treat background as a stronger negative than tool pixels.
- B. Vessel Branch Loss Term: The vessel branch applies combined Dice-IoU loss with boundary-aware constraints for vessel structures.The design emphasizes anatomical region coherence, tool boundaries, and support continuity.
- B. Vessel Branch Loss Term: A boundary-support term constrains tool pixels within vessel pixels using a spatial gradient operator.
- B. Vessel Branch Loss Term: Background is treated as the strongest negative, while tool acts as a structured internal competitor in vessel supervision.
C. Background Branch Loss Term
The background branch is designed to remain discriminative without overwhelming the tool and vessel branches. Its supervision moderates background positives and weights vessel negatives more strongly than tool negatives.
- C. Background Branch Loss Term: The background branch moderates its positive weight to reduce background dominance.
- C. Background Branch Loss Term: Vessel is penalized more strongly than tool because vessel occupies larger contiguous foreground support.
- C. Background Branch Loss Term: Unified scene supervision trains against a multiclass label map representing background, vessel, and tool classes.The objective combines class-balanced cross-entropy and Dice functions to learn global scene context and interclass relationships.
E. Total Loss
Training combines four branch objectives with adaptive uncertainty-based weighting and implements the network across three catheterization datasets. Evaluation uses dataset-specific test sets after repeated training and validation-based model selection.
- E. Total Loss: The total loss adaptively balances tool, vessel, background, and fused-scene objectives using learnable uncertainty-based weights.Learnable scalars rebalance branch losses to reduce gradient interference.
- E. Total Loss: The model uses a modified Xception-style encoder with fused lateral branches and a shared upper decoder trunk for class-specific learning.
- E. Total Loss: Training uses 256 × 256 inputs, Adam optimization, an initial learning rate of 10^-4, batch size 16, and 50 epochs.
- E. Total Loss: The best segmentation-quality model is selected using mean intersection-over-union and evaluated on the corresponding test set for each dataset.
- E. Total Loss: Validation covers three angiogram datasets spanning in-vitro phantom procedures and in-vivo catheterization data.
- E. Total Loss: Datasets are trained and validated separately with masks encoding background, vessel or catheter, and tool classes.
4. Experimental Studies and Evaluations
The experiments evaluate dual-part MLBNet across multiple catheterization datasets using segmentation metrics, visual outputs, and cross-domain transfer. Results show strong background and scene-level performance, but weaker catheter–guidewire discrimination and reduced transfer under heterogeneous domain shifts.
- The evaluation uses accuracy, F-score, and mIoU, including background-excluded variants because dominant background pixels can misinform overall performance.
- 96.94% and 97.28% macro F1 were achieved on Datasets 1 and 2, while CathAction phantom and animal sets reached 66.32% and 77.29%.
- 99.67–99.90% background F1 across test sets highlights severe class imbalance, while foreground metrics reveal weaker performance on catheter–guidewire segmentation.
- Precision of 50.89% and 35.22% and recall of 64.58% and 46.45% indicate high foreground errors for the Dataset 3 phantom and animal angiograms.
- Tool AUPRC reached 0.863 for a camera-based phantom transfer, whereas transfer degraded between heterogeneous animal and phantom domains.
A. Multi-head and Branch-Specific Loss Contributions
Ablations indicate that branch-specific losses and multiple decoder heads jointly improve the stability and class-specific robustness of dual-part MLBNet segmentation. Among decoder outputs, the scene-head branch performed best overall, while single-decoder variants degraded on challenging data.
- Branch-specific loss contributions: The branch-specific three-decoder setup achieved better and more stable segmentation across four datasets, with Dataset 2 approaching categorical cross-entropy performance.The comparison replaced the branch-specific objective with standard categorical cross-entropy in Case #2.
- Multi-head architecture: Single-decoder models retained high performance on simpler Datasets 1–2 but showed clear foreground-segmentation degradation on challenging Dataset 3.The ablation used one decoder with standard categorical cross-entropy.
- Multi-head architecture: The multi-head architecture and branch-specific losses complement each other in supporting robust class-specific learning.
- Decoder-head comparisons: Scene-head predictions performed best across datasets, tool-head predictions followed, and vessel-tool predictions were least stable.Case #4 was similar to Case #1, whereas vessel-tool predictions were least stable.
- Comparison with prior architectures: Dual-part MLBNet models achieved the best metrics among compared architectures and the lowest tool-precision error.Tool-precision error measures false tool predictions divided by total predicted tool pixels; Dataset 3 combines catheter and guidewire as tool.
5. Conclusion and Future Works
The study validated dual-part MLBNet for multi-structure segmentation across catheterization datasets, including in-vitro and in-vivo settings. Future work targets more reliable generalization, broader tool coverage, and explainable outputs for clinical and robotic intervention.
- Conclusion: Dual-part MLBNet was validated for multi-structure segmentation in catheterization scenes using class-aware models across different datasets.
- Conclusion: Evaluation on in-vitro and in-vivo datasets showed stable segmentation and feasibility for cross-domain multi-structure segmentation.
- Future works: Additional runs are needed to examine average performance, while decoder-branch loss consistency and alignment may improve stability and generalization.
- Future works: Future evaluations should include clinical tools such as stents and balloons, alongside explainable outputs for clinical and robot-assisted interventions.