Source-linked AI summary
A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography
Majd Zreik, Robbert W. van Hamersvelt, Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, Ivana Isgum
TL;DR
Different plaque types and stenosis grades affect coronary artery disease management, motivating automated detection and classification. A multi-task RCNN analyzes coronary-artery MPR images using local 3D CNN features and recurrent aggregation, achieving 0.77 accuracy for plaque classification and reporting accurate stenosis-significance classification. The method may support automated triage, although joint classifiers produced physiologically impossible label combinations in fewer than 1.5% of cases.
Problem
Different plaque types and stenosis grades require accurate detection and characterization because they lead to different patient management strategies.
Method
A multi-task RCNN analyzes coronary-artery MPR images by extracting local 3D CNN features and aggregating them with an RNN for simultaneous plaque and stenosis classification.
Results
0.77 accuracy was achieved for plaque detection and characterization, while the method accurately detected and determined the anatomical significance of coronary artery stenosis.
Takeaways & Limitations
The method may enable automated triage of patients without coronary plaque versus those with plaque and stenosis needing further cardiovascular investigation.
Takeaways & Limitations
Joint classification could produce a physiologically impossible combination in fewer than 1.5% of cases because the two softmax classifiers use different parameters.
Abstract
from arXiv · showhide
Various types of atherosclerotic plaque and varying grades of stenosis could lead to different management of patients with coronary artery disease. Therefore, it is crucial to detect and classify the type of coronary artery plaque, as well as to detect and determine the degree of coronary artery stenosis. This study includes retrospectively collected clinically obtained coronary CT angiography (CCTA) scans of 163 patients. To perform automatic analysis for coronary artery plaque and stenosis classification, a multi-task recurrent convolutional neural network is applied on multi-planar reformatted (MPR) images of the coronary arteries. First, a 3D convolutional neural network is utilized to extract features along the coronary artery. Subsequently, the extracted features are aggregated by a recurrent neural network that performs two simultaneous multi-class classification tasks. In the first task, the network detects and characterizes the type of the coronary artery plaque (no plaque, non-calcified, mixed, calcified). In the second task, the network detects and determines the anatomical significance of the coronary artery stenosis (no stenosis, non-significant i.e. <50% luminal narrowing, significant i.e. >50% luminal narrowing). For detection and classification of coronary plaque, the method achieved an accuracy of 0.77. For detection and classification of stenosis, the method achieved an accuracy of 0.80. The results demonstrate that automatic detection and classification of coronary artery plaque and stenosis are feasible. This may enable automated triage of patients to those without coronary plaque and those with coronary plaque and stenosis in need for further cardiovascular workup.
I. INTRODUCTION
Coronary plaque composition and stenosis severity influence patient management, but current CCTA assessment is variable or cumbersome. The study proposes a multi-task RCNN to jointly classify plaque type and stenosis significance from coronary-artery MPR images.
- Different plaque types and stenosis grades require accurate detection and characterization because they lead to different patient management strategies.
- CCTA enables noninvasive plaque characterization and stenosis grading, but visual assessment has substantial interobserver variability.
- Semi-automatic approaches relying on lumen and arterial-wall segmentation are time-consuming and cumbersome, particularly with extensive plaque or imaging artefacts.
- Automatic methods have commonly targeted either calcified or non-calcified plaque, while comprehensive plaque classification remains challenging.
- The proposed multi-task RCNN analyzes sequences of small volumes along the coronary centerline to jointly classify plaque type and stenosis significance.A 3D CNN extracts features from local volumes, and an RNN aggregates them along the plaque.
A. Patient and image data
The study used retrospectively collected clinical CCTA scans from 163 patients acquired at one hospital between 2012 and 2016. Coronary centerlines were extracted and used to reconstruct straightened isotropic MPR images for analysis.
- The dataset comprised retrospectively collected clinical CCTA scans from 163 patients acquired between 2012 and 2016.Patients had a mean age of 59.2 ± 8.8 years, and 126 were male.
- Images were reconstructed with 0.9 mm slice thickness, 0.45 mm slice increment, and 0.38–0.56 mm in-plane resolution.
- A previously described centerline-extraction method required manual placement of one seed point in the artery of interest.
- Extracted centerlines were used to reconstruct 0.3 mm isotropic straightened MPR images for each artery.
B. Reference standard
The reference standard used expert annotations on coronary-artery MPR images, labeling plaque type and stenosis significance for clinically evaluated arteries. Segment labels were also translated to artery- and patient-level assessments.
- Only coronary arteries larger than 1.5 mm were annotated, reflecting the clinical evaluation threshold used with CCTA.
- An expert marked each plaque’s start and end points, composition, and stenosis significance using SCCT reporting guidelines.Stenosis significance was visually estimated from maximal luminal narrowing.
- Plaque voxels were not segmented; instead, the artery segment containing the plaque was identified.
- The dataset contained 1,259 manually labeled segments in 534 arteries, including plaque-free segments and plaques causing significant or non-significant stenosis.
- A second trained observer independently annotated the same arteries while blinded to the reference standard.
III. METHOD
The method uses a multi-task recurrent convolutional neural network on sequential 3D volumes extracted along coronary artery centerlines in MPR images. A CNN extracts local features, an RNN aggregates them, and two classifiers jointly predict plaque and stenosis categories.
- III. METHOD: The RCNN analyzes sequential local artery volumes so image features can be extracted from small regions and aggregated along the plaque.
- III. METHOD: The workflow extracts coronary centerlines, reconstructs stretched MPR images, and applies two simultaneous multi-class classification tasks.
- III. METHOD: The network classifies plaque as no plaque, non-calcified, mixed, or calcified.
- III. METHOD: The network classifies stenosis as no stenosis, non-significant, or significant.
- III. METHOD: A 3D CNN processes 25 × 25 × 25-voxel cubes, an RNN with GRUs processes the sequence, and two softmax classifiers produce the outputs.
IV. EVALUATION
Performance was evaluated at segment, artery, and patient levels using label-overlap criteria tailored to plaque and stenosis tasks. Plaque reliability was summarized with accuracy, unweighted F1, and Cohen’s κ, while observer agreement used the same metrics.
- Plaque and stenosis performance was evaluated at segment-, artery-, and patient-levels, with complete centerlines used for artery- and patient-level analyses.
- A plaque or stenosis label was a segment-level true positive when it overlapped a same-label reference segment by at least 1 mm.Absence labels required no point along the segment to contain plaque or stenosis.
- Plaque evaluation used average segment accuracy, unweighted mean F1 across labels, and unweighted Cohen’s κ against the reference standard.
- Stenosis significance was considered correctly detected at artery or patient level when any predicted centerline label matched the reference label.
- Interobserver reliability was assessed with the same metrics by comparing a second observer’s annotations with the reference standard.
V. EXPERIMENTS AND RESULTS
The experiments trained and tested the multi-task network on retrospectively collected CCTA scans, using augmentation, stratified sampling, joint losses, and centerline-based inference. Testing classified points along complete analyzed arteries rather than predefined segments.
- 163 patients were split into 81 training, 17 validation, and 65 testing scans, with hyperparameters selected using training and validation data only.
- Data augmentation randomized rotations around the artery centerline and varied sequence centers to improve invariance to orientation and annotation inaccuracies.
- Stratified sampling addressed unbalanced plaque and stenosis labels during training, reducing potential bias toward common categories.
- The network minimized the average of plaque and stenosis categorical cross-entropy losses with L2 regularization.The regularization coefficient was γ = 0.001 for all layers.
- Training used Adam with learning rate 0.001, mini-batches of 36 sequences, 50% recurrent-layer dropout, and 50,000 iterations.
- During testing, fixed-length sequences centered on every coronary centerline point produced labels along the whole analyzed artery.
A. Plaque detection and characterization
The proposed RCNN detected plaque and classified it into four categories at the segment level. Multi-class plaque characterization reached 0.77 accuracy, while binary plaque detection reached 0.85 accuracy.
- The plaque task classified segments as no plaque, non-calcified, mixed, or calcified plaque.
- Figure 4 compares training and validation losses and validation accuracies with and without data augmentation for both plaque and stenosis tasks.
- 0.85 segment-level accuracy was achieved for detecting plaque of any type versus no plaque.The second observer achieved 0.90 accuracy on this binary task.
- 0.77 segment-level accuracy and unweighted κ of 0.61 were achieved for detecting and characterizing plaque across four classes.The second observer achieved 0.80 accuracy and unweighted κ of 0.67.
- Plaque characterization accuracy varied by artery, reaching 0.81 in LCX and 0.72 in RCA.
B. Stenosis detection and characterization
The stenosis task evaluated detection and anatomical significance classification across segment-, artery-, and patient-levels. The figures illustrate class probabilities, whole-artery overlays, and a bifurcation-related false stenosis detection.
- Stenosis results were analyzed at segment-, artery-, and patient-levels using confusion matrices.
- Figure 5 displays plaque and stenosis class probabilities against manually annotated boundaries and reference labels.
- A mixed plaque with significant stenosis was classified as calcified plaque with significant stenosis in one example.
- A stenosis was falsely detected distal to an artery bifurcation where the lumen diameter suddenly decreased.
- Figure 6 overlays predicted plaque and stenosis labels across an entire artery, using distinct colors for no, non-significant, and significant stenosis.The overlays were extended to 25 × 25 voxels around the artery centerline for illustration.
C. Impact of the RCNN architecture
Replacing the recurrent layers with fully connected layers provided a parameter-matched comparison for the proposed RCNN. The recurrent architecture achieved segment-, artery-, and patient-level stenosis accuracies of 0.80, 0.76, and 0.75, respectively.
- Architecture comparison: The comparison network retained the CNN architecture but replaced the RNN with global max pooling and fully connected layers.The alternative used two fully connected layers after pooling CNN-extracted features from the input sequence.
- Stenosis results: 0.80, 0.76, and 0.75 accuracy were obtained for stenosis characterization at the segment-, artery-, and patient-levels, respectively.The corresponding linearly weighted κ values were 0.68, 0.66, and 0.67.
- Parameter matching: 341,191 parameters were used in the fully connected comparison network versus 340,295 in the RCNN network.The unit count was increased from 64 to 192 to match the trainable-parameter totals.
D. Single vs. multi-task classification
The study compared joint plaque-and-stenosis classification with a single seven-class formulation and with CNN-only alternatives. The comparisons address whether separate multi-task outputs and recurrent aggregation improve classification performance.
- Single-task classification: A single-task formulation combines plaque and stenosis into seven output classes, whereas the proposed approach treats them as two classification tasks.The seven classes distinguish no plaque, plaque types with non-significant stenosis, and plaque types with significant stenosis.
- Experimental comparison: The experiments compared recurrent and CNN-only architectures using either one softmax classifier or two softmax classifiers.All comparison networks used the same training, validation, and test image sets as the proposed network.
- Stenosis detection: 0.61 sensitivity and 0.65 PPV were achieved for detecting significant stenosis.The cited prior methods reported sensitivities of 0.50 and 0.28 and PPVs of 0.27 and 0.23, respectively.
- CNN baseline: The CNN baseline processes the sequence through global max pooling and fully connected layers before two softmax classifiers.The dense layers use 196 units each in this parameter-matched baseline.
- Comparison boundary: Direct comparison with prior plaque methods is not feasible because those methods quantify plaque at voxel level or require manual or semi-automatic interaction.The proposed method does not provide voxel-level quantification.
VI. DISCUSSION AND CONCLUSION
The RCNN analyzes centerline-based MPR volumes to jointly characterize plaque and stenosis without arterial lumen segmentation. Results support feasible automatic classification, while annotation granularity, bifurcations, artifacts, and task consistency constrain interpretation.
- Method: The RCNN uses a 3D CNN followed by an RNN to classify plaque type and stenosis significance from MPR views along the coronary centerline.Unlike many prior methods, it requires the CCTA image and extracted centerline but not coronary lumen segmentation.
- Plaque characterization: Plaque classification was accurate but had moderate reliability, with mixed plaque remaining difficult to distinguish from calcified and non-calcified plaque.The ambiguity arises because mixed plaque contains both calcified and non-calcified components.
- Stenosis characterization: Stenosis significance classification achieved excellent reliability and approached the performance of the second observer.Patients with anatomically significant stenosis were detected with high accuracy in the reported experiments.
- Recurrent aggregation: The recurrent architecture had a clear advantage over a network without recurrent units.The authors relate this to sequentially aggregating information from several volumes along the coronary artery rather than analyzing only a local volume.
- Segmentation requirement: Using only the centerline avoids potential error propagation from arterial lumen segmentation, which can require substantial manual interaction in heavily calcified arteries.The method therefore omits a segmentation step used by many previous approaches.
- Multi-task trade-off: Jointly performing plaque and stenosis classification halved inference time to 1.8 seconds per artery on average but allowed physiologically impossible mismatched outputs in fewer than 1.5% of cases.A single-task network could prevent this scenario, although the multi-task approach performed better in the reported comparisons.
- Reference standard: The segment-level reference standard assigned one label to an entire plaque-containing segment, limiting agreement for automatically identified plaque and stenosis boundaries.Future annotations could instead label each arterial-wall voxel or lumen cross-section.
- Conclusion: The study concludes that automatic plaque and stenosis characterization may support triage between patients without plaque and those needing further cardiovascular investigation.The conclusion is limited to the proposed algorithm and its evaluated classification tasks.