Source-linked AI summary
Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction
Botong Wu, Zhen Zhou, Jianwei Wang, Yizhou Wang
TL;DR
CNN-derived features in prior nodule classifiers are difficult for radiologists to understand, making predictions hard to accept without related imaging findings. PN-SAMP is an end-to-end multi-task 3D CNN that predicts malignancy, learns nodule attributes, segments nodules, and uses multiple window widths and centers. Extensive experiments showed that the proposed model achieved 97.58% for nodule malignancy prediction, 89.33% for attributes prediction, and 73.89% for nodule segmentation.
Problem
CNN-derived features in prior nodule classifiers are difficult for radiologists to understand, making predictions hard to accept without related imaging findings.
Source: §1 · p. 1Method
PN-SAMP is an end-to-end multi-task 3D CNN that predicts malignancy, learns nodule attributes, segments nodules, and uses multiple window widths and centers.
Source: Contributions · p. 2Results
Extensive experiments showed that the proposed model achieved 97.58% for nodule malignancy prediction, 89.33% for attributes prediction, and 73.89% for nodule segmentation.
Source: §4 · p. 4Takeaways & Limitations
The model's semantic attributes and nodule regions are intended to help radiologists assess malignancy predictions and understand the model's decisions.
Source: Contributions · p. 2, §4 · p. 4The authors identify malignancy prediction with pathology and unsure-labeled data, and further network analysis for interpretability, as future work.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Refer to the literature of lung nodule classification, many studies adopt Convolutional Neural Networks (CNN) to directly predict the malignancy of lung nodules with original thoracic Computed Tomography (CT) and nodule location. However, these studies cannot tell how the CNN works in terms of predicting the malignancy of the given nodule, e.g., it's hard to conclude that whether the region within the nodule or the contextual information matters according to the output of the CNN. In this paper, we propose an interpretable and multi-task learning CNN -- Joint learning for \textbf{P}ulmonary \textbf{N}odule \textbf{S}egmentation \textbf{A}ttributes and \textbf{M}alignancy \textbf{P}rediction (PN-SAMP). It is able to not only accurately predict the malignancy of lung nodules, but also provide semantic high-level attributes as well as the areas of detected nodules. Moreover, the combination of nodule segmentation, attributes and malignancy prediction is helpful to improve the performance of each single task. In addition, inspired by the fact that radiologists often change window widths and window centers to help to make decision on uncertain nodules, PN-SAMP mixes multiple WW/WC together to gain information for the raw CT input images. To verify the effectiveness of the proposed method, the evaluation is implemented on the public LIDC-IDRI dataset, which is one of the largest dataset for lung nodule malignancy prediction. Experiments indicate that the proposed PN-SAMP achieves significant improvement with respect to lung nodule classification, and promising performance on lung nodule segmentation and attribute learning, compared with the-state-of-the-art methods.
1. INTRODUCTION
Pulmonary nodule CAD should provide malignancy predictions alongside evidence such as high-level attributes and nodule segmentation, but CNN-derived features can be difficult for radiologists to understand. PN-SAMP combines multi-window CT inputs with attribute learning and segmentation; the introduction reports significant classification improvement and promising results on the other tasks.
1. INTRODUCTION
- CNN-derived features in prior nodule-classification systems are difficult for radiologists to understand, complicating acceptance of predictions without related imaging findings.The stated CAD requirements include malignancy diagnosis and related evidence such as high-level attributes and nodule-area segmentation.Source: §1 · p. 1
- On LIDC-IDRI, the introduction reports significant improvement in pulmonary nodule classification and promising performance for segmentation and attribute learning.The passage names M2LSCS in its experimental claim, while the proposed method elsewhere is called PN-SAMP.Source: §1 · p. 1
- PN-SAMP combines two CT patches made with different window widths and centers as channels, then learns attributes and malignancy alongside nodule segmentation.Attribute learning and malignancy prediction connect both to semantic information and to a sub-3DCNN that takes the segmentation as input.Source: Figure 1 · p. 2
Contributions
The contributions are an end-to-end multi-task network for malignancy prediction, nodule segmentation, and attribute learning; multi-window CT inputs; and reported best performance across the three tasks on LIDC-IDRI.
Contributions
- PN-SAMP is an end-to-end multi-task 3D CNN that simultaneously predicts malignancy, segments nodule areas, and learns nodule attributes.Its semantic attributes and nodule regions are intended to help radiologists understand its decisions.Source: Contributions · p. 2
- PN-SAMP uses multiple window widths and centers to enrich nodule information, and the authors state that multi-task learning improves each task's performance.Source: Contributions · p. 2
- The authors report that PN-SAMP achieves the best performance in segmentation, attribute learning, and malignancy prediction on LIDC-IDRI compared with prior state-of-the-art methods.Source: Contributions · p. 2
2. MATERIALS
The evaluation uses LIDC-IDRI scans and radiologist ratings for nodule malignancy and attributes, with nodules rated at a mean malignancy score of 3 excluded. Each nodule is represented by a normalized 64 × 64 × 64 × 2 image volume.
2. MATERIALS
- Evaluation is performed on the LIDC-IDRI dataset, which includes 1,010 patients (1,018 scans) and 2,660 nodules; slice thickness varies from 0.45 mm.Source: §2 · p. 2
- Radiologists rate nine nodule attributes on a 1-to-5 scale; nodules with a mean malignancy rating of 3 are excluded, leaving 898 benign and 506 malignant nodules.Ratings are averaged across radiologists, and each nodule is extracted as a 64 × 64 × 64 × 2 volume normalized to 0.6 mm along each dimension.Source: §2 · p. 2
3. METHODS
PN-SAMP links nodule segmentation, attribute learning, and malignancy prediction in a multi-task network. The reported comparisons show competitive classification, improved segmentation over 3D-UNet, and gains for multi-task learning over single-task models.
3.1. Multi-Task Deep Convolutional Neural Network
- A reduced 3D-UNet segments nodules; attribute prediction combines semantic and segmentation features, and malignancy prediction builds on attribute features.The attribute/classification design also lets semantic-label gradients propagate to the segmentation subnet.Source: §3.1 · pp. 2–3
3.2.1. Experimental Settings
- Evaluation used five-fold cross-validation on 1,404 malignant and benign volumes, multiple WW/WC inputs, scratch training, and single-patch inference under 0.5 seconds on TITAN X Pascal.The training setup used Adam with a learning rate of 1e−3 and reserved 10% of volumes for validation.Source: §3.2.1 · p. 3
3.2.3. Evaluation on Segmentation
- Proposed variants outperform 3D-UNet by an average 1.99% in Dice coefficient while using 1.5M rather than 16M parameters.Table 1 compares segmentation Dice coefficient and standard error of the mean across the proposed WW/WC variants and U-Net.Source: Table 1 · p. 3, §3.2.3 · p. 3
3.2.2. Evaluation Metric
- Segmentation uses Dice coefficient; attribute and malignancy ratings use off-by-one accuracy, counting ±1 as acceptable.Source: §3.2.2 · p. 3
3.2.4. Evaluation on Classification
- PN-SAMP-S1/S2 are comparable to or better than prior studies, while multiple WW/WC gives PN-SAMP-M the best classification performance.The nine-attribute evaluation, including malignancy, reports 89.33% accuracy.Source: Table 2 · p. 3, §3.2.4 · p. 4
3.2.5. Evaluation on Different Tasks
- The multi-task model exceeds the single-task segmentation and classification models by 3.69% and 4.69%.The authors attribute the gains to additional shape and semantic supervision and to information flow between segmentation and classification subnetworks.Source: §3.2.5 · p. 4, Table 3 · p. 4
4. DISCUSSION AND CONCLUSION
PN-SAMP combines end-to-end multi-task prediction with nodule segmentation and attributes alongside malignancy ratings. The reported scores were 97.58% for malignancy prediction, 89.33% for attributes prediction, and 73.89% for segmentation.
- PN-SAMP is an end-to-end multi-task 3D convolutional neural network for pulmonary nodule segmentation, attributes, and malignancy prediction, proposed to address model interpretability.Source: §4 · p. 4
- 97.58% for nodule malignancy prediction, 89.33% for attributes prediction, and 73.89% for nodule segmentation were reported for the proposed model.Source: §4 · p. 4
- Future work will examine pathology data and unsure-labeled data for malignancy prediction, and analyze neural networks to further increase interpretability.Source: §4 · p. 4
- Fig. 2 places malignant nodules above benign ones, compares predictions (A) with ground truth (B), and marks within-one ratings blue and out-of-range ratings red.Source: Figure 2 · p. 4