PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification

Wentao Zhu, Chaochun Liu, Wei Fan, Xiaohui Xie

arXiv:1801.09555v1cs.CVcs.LGcs.NE

TL;DR

DeepLung addresses the need for a fully automated lung CT cancer diagnosis system that integrates nodule detection and classification. It uses 3D dual path networks with 3D Faster R-CNN for detection and GBM-based classification, and reports performance comparable to experienced doctors at both nodule and patient levels.

  • Problem

    Prior lung CT research mainly addressed nodule detection or classification separately, leaving few complete systems for fully automated lung CT cancer diagnosis.

  • Method

    DeepLung combines 3D dual path networks for detection and classification, using 3D Faster R-CNN for candidate nodules and GBM with deep features, nodule size, and raw pixels for classification.

  • Results

    DeepLung is reported as comparable to experienced doctors for nodule-level and patient-level diagnosis, while its classification framework surpasses experienced doctors on LIDC-IDRI.

  • Takeaways & Limitations

    The integrated DeepLung system provides fully automated lung CT diagnosis by detecting candidate nodules, classifying them, and fusing nodule results for patient-level diagnosis.

  • Takeaways & Limitations

    The analysis notes that human doctors may be limited by processing low-signal 3D CT data while observing only one slice at a time.

Abstract

from arXiv · show

In this work, we present a fully automated lung computed tomography (CT) cancer diagnosis system, DeepLung. DeepLung consists of two components, nodule detection (identifying the locations of candidate nodules) and classification (classifying candidate nodules into benign or malignant). Considering the 3D nature of lung CT data and the compactness of dual path networks (DPN), two deep 3D DPN are designed for nodule detection and classification respectively. Specifically, a 3D Faster Regions with Convolutional Neural Net (R-CNN) is designed for nodule detection with 3D dual path blocks and a U-net-like encoder-decoder structure to effectively learn nodule features. For nodule classification, gradient boosting machine (GBM) with 3D dual path network features is proposed. The nodule classification subnetwork was validated on a public dataset from LIDC-IDRI, on which it achieved better performance than state-of-the-art approaches and surpassed the performance of experienced doctors based on image modality. Within the DeepLung system, candidate nodules are detected first by the nodule detection subnetwork, and nodule diagnosis is conducted by the classification subnetwork. Extensive experimental results demonstrate that DeepLung has performance comparable to experienced doctors both for the nodule-level and patient-level diagnosis on the LIDC-IDRI dataset.\footnote{https://github.com/uci-cbcl/DeepLung.git}

1. Introduction

DeepLung addresses the limited availability of complete automated lung CT diagnosis systems by combining deep 3D detection and classification networks. Its integrated system reports performance comparable to experienced doctors for nodule-level and patient-level diagnosis.

  • Motivation: DeepLung targets fully automated lung CT cancer diagnosis by integrating nodule detection and nodule classification.The paper describes this as a gap in prior work, which mainly treated detection and classification separately.
  • Classification: For classification, GBM combines deep 3D dual path features with nodule size and cropped raw CT pixels.The feature design spans learned representations and nodule-level measurements or image data.
  • System framework: The system first detects candidate nodules with 3D Faster R-CNN, then classifies them as malignant or benign and fuses nodule results for patient-level diagnosis.This is the operational sequence described for a complete CT diagnosis.
  • Architecture: DeepLung uses 3D dual path networks for both detection and classification to exploit volumetric CT data while keeping the networks compact.The detection network combines 3D Faster R-CNN, dual path blocks, and a U-net-like encoder-decoder structure.
  • Results: DeepLung’s complete system is reported as comparable to experienced doctors for both nodule-level and patient-level diagnosis on LIDC-IDRI.The classification framework is also reported to outperform state-of-the-art approaches and experienced doctors on the public dataset.

2. Related Work

Prior lung CT research developed separate detection and classification methods, increasingly using deep networks for both tasks. DeepLung differs by combining 3D dual path features with multi-granularity inputs for nodule and patient-level diagnosis.

  • Nodule detection: Earlier lung nodule detection methods used hand-designed features, while later approaches applied deep ConvNets to generate candidate regions and reduce false positives.The related work describes both traditional feature engineering and deep-learning detection pipelines.
  • Nodule classification: Nodule classification research progressed from segmentation and manual features to deep networks, including multi-scale, transfer-learning, and multi-instance methods.The cited approaches address nodule-level or patient-level diagnosis using different learned representations.
  • Dual path networks: Dual path connections combine residual learning for feature reuse with dense connections for exploiting new features.The connection divides feature maps between the two pathways.
  • DeepLung approach: DeepLung applies a 3D DPN to raw CT nodules and uses GBM with DPN features, raw pixels, and nodule size for diagnosis.Patient-level diagnosis is obtained by fusing nodule-level diagnoses.

3. DeepLung Framework

DeepLung combines a 3D Faster R-CNN detector with a GBM-based classifier using deep 3D DPN features and additional nodule information. Its design uses dual-path 3D networks, encoder-decoder detection, and sequential candidate detection and diagnosis.

  • System overview: DeepLung combines 3D Faster R-CNN nodule detection with GBM classification using deep 3D DPN features, nodule size, and raw CT pixels.The detected nodules are passed to the classification stage for diagnosis.
  • 3D dual path design: Dual path connections split feature maps between dense connection and residual learning to reuse features while exploiting new ones.This design integrates residual feature reuse with dense extraction of new features.
  • Nodule detection: The detection network uses a U-net-like encoder-decoder with 3D dual path blocks to learn nodule features from cropped 96 × 96 × 96 CT volumes.The encoder uses eight dual path blocks after initial convolutional layers, while GPU memory limits motivate cropped inputs.
  • Nodule detection: The detector uses three anchors and multi-task learning for candidate-box classification plus nodule coordinate and diameter regression.Anchor labels are based on IoU with ground-truth boxes, with positive anchors above 0.5 IoU and negative anchors below 0.02 IoU.
  • Nodule classification: The classification network crops 32 × 32 × 32 volumes around detected nodules, applies 30 3D dual path blocks, and diagnoses benign or malignant nodules.The system also retains detected nodule size and combines learned features with raw cropped pixels for GBM classification.
  • Nodule classification: Combining nodule size and raw cropped CT pixels with GBM achieved 86.12% average test accuracy, while the full constructed feature achieved the best diagnosis performance.The constructed feature concatenates 2,560-dimensional deep 3D DPN features with nodule size and raw 3D cropped pixels.

4. Experiments

Experiments evaluate DeepLung’s detection, classification, and fully automated diagnosis using patient-level splits from LUNA16 and LIDC-IDRI. The system achieves strong detection and classification performance, including results comparable to experienced doctors at nodule and patient levels.

  • Datasets and evaluation: Experiments use LUNA16 for detection and LIDC-IDRI annotations for classification, with patient-level cross-validation splits.LUNA16 contains 888 low-dose CTs, while LIDC-IDRI contains 1,018; the classification evaluation uses 1,004 nodules, including 450 positive cases.
  • Datasets and evaluation: The detector is trained with 10-fold patient-level cross-validation and evaluated using FROC, averaging recall across seven false-positive rates per scan.The evaluation points are 0.125, 0.25, 0.5, 1, 2, 4, and 8 false positives per scan.
  • Nodule detection: 3D DPN26 Faster R-CNN achieves 84.2% FROC without false-positive reduction, exceeding the previous 83.9% obtained with two-stage training.It uses only one quarter of the parameters of the 3D Res18 Faster R-CNN while performing better than that baseline.
  • Nodule classification: The classification model achieves 90.44% accuracy using deep 3D dual path features together with detected nodule size and raw nodule pixels.The reported performance is better than Multi-scale CNN, Vanilla 3D CNN, and Multi-crop CNN.
  • Fully automated diagnosis: For fully automated diagnosis, DeepLung obtains 81.42% accuracy on detected true-positive nodules and reduces 97.02% of detected false positives.The true-positive accuracy exceeds the experienced doctors’ average of 78.36%. Patient-level accuracy is 81.41%, reported as 99% of the doctors’ average.

5. Discussion

DeepLung’s visualizations show accurate nodule detection and classification on test fold 1, including cases where doctors’ annotations were incorrect. The authors suggest that processing full 3D CT data may help address limitations of observing individual slices.

  • Nodule Detection: Figure 7 compares ground-truth nodules with detections using central-slice images, circle sizes, slice numbers, and detection probabilities.Red circles mark ground truths, blue circles mark detections, and circle diameter reflects nodule size.
  • Nodule Detection: Detected nodule positions, central slices, circle sizes, and high detection probabilities closely matched ground truths in test fold 1.The authors conclude that 3D Faster R-CNN detects nodules effectively.
  • Nodule Classification: Figure 8 includes nodules correctly classified by DeepLung but incorrectly annotated by some doctors; the first seven are benign and the remainder malignant.Displayed values are predicted malignant probabilities followed by the incorrect doctor annotation.
  • Nodule Classification: The authors observe that doctors misdiagnosed some nodules, potentially because weak irregular boundaries or normal tissues were difficult to interpret in low-signal-to-noise 3D CT data.They also mention internal bias and the limitation of observing only one slice at a time.
  • Nodule Classification: Machine-learning methods can process all input slices together and learn high-dimensional features, which the authors suggest may support more consistent and accurate diagnoses.This is presented as a potential use of DeepLung for assisting doctors.

6. Conclusion

DeepLung is a fully automated deep-learning system for lung CT cancer diagnosis that detects candidate nodules and classifies them as benign or malignant. It combines 3D dual path networks with 3D Faster R-CNN detection and gradient-boosting classification, with experiments on LUNA16 and LIDC-IDRI demonstrating superior system performance.

  • System Overview: DeepLung combines nodule detection and classification into a fully automated lung CT cancer diagnosis system.Candidate nodules are detected first, then classified as benign or malignant.
  • Network Design: The system uses two deep 3D convolutional networks based on compact 3D dual path networks to exploit the three-dimensional structure of CT images.The authors state that dual path networks are more compact and can perform better than residual networks.
  • Nodule Detection: For detection, DeepLung uses 3D Faster R-CNN with 3D dual path blocks and a U-net-like encoder-decoder structure.The detected nodules are subsequently passed to the classification network.
  • Nodule Classification: For classification, DeepLung extracts features with a deep 3D dual path network and trains a gradient boosting machine to classify candidate nodules.The classifier uses combined features for benign-versus-malignant prediction.
  • Evaluation: Extensive experiments on the LUNA16 and LIDC-IDRI datasets demonstrate superior performance of the DeepLung system.
Loading 1801.09555v1…