Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation

Holger R. Roth; Le Lu; Jiamin Liu; Jianhua Yao; Ari Seff; Kevin Cherry; Lauren Kim; Ronald M. Summers

Source-linked AI summary

Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation

Holger R. Roth, Le Lu, Jiamin Liu, Jianhua Yao, Ari Seff, Kevin Cherry, Lauren Kim, Ronald M. Summers

arXiv:1505.03046v2May 12, 2015cs.CV

TL;DR

Medical CADe methods can achieve high sensitivity while producing many false positives. The paper adds a ConvNet-based coarse-to-fine second tier that classifies random 2D or 2.5D views of candidates and aggregates their predictions. Across three CT applications, this approach markedly improved sensitivity at 3 FPs per patient.

Problem
CADe methods often achieve high sensitivity at the cost of high false-positive rates per patient.
Method
The method uses a two-tiered CADe cascade in which ConvNets classify and aggregate random 2D or 2.5D candidate views sampled across scales, translations and rotations.
Results
57% to 70%, 43% to 77% and 58% to 75%: sensitivity at 3 FPs per patient improved for sclerotic metastases, lymph nodes and colonic polyps, respectively.
Takeaways & Limitations
Random ConvNet observations improved CADe sensitivity across clinically relevant false-positive rates in three CT pathology categories.
Takeaways & Limitations
For 6–9 mm polyps, random 2.5D view sampling may be suboptimal because sampled views can contain only tiny portions of the target.

Abstract

from arXiv · show

Automated computer-aided detection (CADe) in medical imaging has been an important tool in clinical practice and research. State-of-the-art methods often show high sensitivities but at the cost of high false-positives (FP) per patient rates. We design a two-tiered coarse-to-fine cascade framework that first operates a candidate generation system at sensitivities of $\sim$100% but at high FP levels. By leveraging existing CAD systems, coordinates of regions or volumes of interest (ROI or VOI) for lesion candidates are generated in this step and function as input for a second tier, which is our focus in this study. In this second stage, we generate $N$ 2D (two-dimensional) or 2.5D views via sampling through scale transformations, random translations and rotations with respect to each ROI's centroid coordinates. These random views are used to train deep convolutional neural network (ConvNet) classifiers. In testing, the trained ConvNets are employed to assign class (e.g., lesion, pathology) probabilities for a new set of $N$ random views that are then averaged at each ROI to compute a final per-candidate classification probability. This second tier behaves as a highly selective process to reject difficult false positives while preserving high sensitivities. The methods are evaluated on three different data sets with different numbers of patients: 59 patients for sclerotic metastases detection, 176 patients for lymph node detection, and 1,186 patients for colonic polyp detection. Experimental results show the ability of ConvNets to generalize well to different medical imaging CADe applications and scale elegantly to various data sets. Our proposed methods improve CADe performance markedly in all cases. CADe sensitivities improved from 57% to 70%, from 43% to 77% and from 58% to 75% at 3 FPs per patient for sclerotic metastases, lymph nodes and colonic polyps, respectively.

I. INTRODUCTION

The paper addresses high false-positive rates in CADe by adding a ConvNet-based second tier to existing candidate-generation systems. Random 2D or 2.5D observations are aggregated to improve detection across three CT applications.

CADe can reduce radiologists’ workload because manually reviewing thousands of candidate regions and every CT section is time-consuming and error-prone.
Traditional CADe commonly uses hand-crafted image features with binary or discrete classifiers to distinguish normal from abnormal anatomy.
The proposed hierarchical two-tiered system builds on existing CADe to combine high recall with low or manageable false-positive rates per patient.
Random 2D or 2.5D ConvNet observations replace the non-parametric component of a hybrid coarse-to-fine system by aggregating classification decisions.
57% to 70%, 43% to 77% and 58% to 75%: sensitivities at 3 FPs per patient improved for sclerotic metastases, lymph nodes and colonic polyps, respectively.The evaluation covered three datasets and CADe applications, including 1,186 patients for the colonic-polyp dataset.

II. METHODS

The methods introduce ConvNets for extracting learned image features and classifying 2D or 2.5D medical-image inputs. Convolution, pooling, learned filters, and regularization form the core processing pipeline.

A. Convolutional Neural Networks: ConvNets compute image features with convolutional filters whose kernel elements are learned from labeled examples rather than hand-crafted.
A. Convolutional Neural Networks: Fig. 1 applies a ConvNet to a 2.5D CT volume of interest and specifies filters, kernel sizes, connections, and overlapping max-pooling with stride 2.
A. Convolutional Neural Networks: Max-pooling summarizes neighboring feature responses and supports spatial invariance, while later locally connected and fully connected layers perform classification.
A. Convolutional Neural Networks: Convolutional filters produce feature responses by sliding kernels over the input region of interest, with padding available to preserve response size.
A. Convolutional Neural Networks: The first-layer trained filters produce response patterns for imaging data, including sclerotic bone lesions, and are learned automatically for each classification task.
A. Convolutional Neural Networks: A final softmax layer provides a classification probability, while DropConnect regularizes fully connected layers by preventing unit co-adaptation.

B. Applying ConvNets to CADe – a 2D or 2.5D Approach

The method represents CADe candidates as 2D or 2.5D observations and aggregates ConvNet predictions from randomly transformed views to produce per-candidate probabilities.

2D and 2.5D representations: CADe candidates are sampled as 2D ROIs or 2.5D VOIs, with candidate locations typically generated at near-100% sensitivity but high false-positive rates.The cited examples report approximately 40–60 false positives for lymph-node or bone-lesion data and approximately 150 for colonic-polyp cases.
Random observations: Random observations vary each ROI or VOI through physical scale, translations, and rotations, yielding N = Ns × Nt × Nr views.Translations use a random vector v, rotations use a random angle α, and fixed pixel dimensions are maintained through resampling.
Random observations: Multiple observations expand training data and help avoid overfitting while supporting 2D or 2.5D sampling for each candidate.When slice thickness or inter-slice distance is at least 5 mm, only 2D axial reformatting and sampling is employed.
Prediction aggregation: At testing, the ConvNet probabilities from N random observations are averaged at each ROI to obtain a final per-candidate probability.The individual terms Pi(x) are classification probabilities for single 2D or 2.5D image patches.

D. Candidate Generation

Candidate generation supplies high-sensitivity lesion locations for the second-stage ConvNet classifier, using existing CADe systems across spine, lymph-node, and colonic-polyp tasks.

Candidate-generation framework: Any CADe system with approximately 95% sensitivity and no more than 150 false positives per patient can provide candidate locations for the proposed second-stage classifier.The framework labels generated candidates as positive or negative examples for ConvNet training in a coarse-to-fine cascade.
Applications: Three existing systems generate candidates for sclerotic spine metastases, lymph nodes, and colonic polyps.The lymph-node systems cover mediastinal and abdominal regions.
Applications: The spine system segments vertebrae and identifies detections with higher mean attenuation than neighboring subsegments.Watershed-based vertebral subdivision and subsequent merging address the candidate-generation process.
Applications: The lymph-node systems use anatomical or shape information before SVM-based candidate classification in mediastinal and abdominal regions.The cited passage specifically describes voxel-level shape features and spatial priors for mediastinal detection.
Applications: The colonic-polyp system segments the colon, removes tagged fluid, and generates candidates from local surface shape features.The approach uses features such as mean curvature and sphericity and is paired with random-view ConvNet aggregation for false-positive rejection.

E. Cascaded CADe Architectures for False Positive Reduction

Cascaded CADe systems reduce false positives either by retraining classifiers with new image features or by adding application-dependent postfilters.

Cascade types: One cascade type extracts new image features and retrains a classifier on all candidates.This approach is represented by several cited CADe systems using additional candidate-level information.
Cascade types: A second cascade type designs application-dependent post-filtering components to remove specific non-lesion structures.Examples include flexible tubes, the ileo-cecal valve, and extra-colonic findings in CT colonography.
Cascade rationale: Additional image features are calculated per candidate because brute-force search during candidate generation is no longer necessary.The cited passage frames these features as a way to reveal information omitted from the candidate-generation step.

A. Imaging Data Sets and Implementation

The study evaluates ConvNet-based CADe across three CT datasets and compares 2D, 2.5D, and 3D configurations, including training-time and data-augmentation effects.

The evaluation covers sclerotic metastases, lymph nodes, and colonic polyps across datasets containing 59, 176, and 1,186 patients, respectively.
Training times range from 9–12 hours for lymph nodes to 37 hours for colonic polyps, while classifying one CT volume with N = 100 observations takes circa 1–5 minutes.
The first convolutional layer learns task-specific filters, using one-channel axial inputs for metastases and three-channel orthogonal views for lymph nodes and polyps.
The comparison evaluates 2D ROIs, proposed 2.5D VOIs, and 3D VOI stacks, with data augmentation tested on abdominal lymph nodes.
3D augmentation improves performance but requires ∼4× more training time and performs only comparably to augmented 2.5D.

D. Detection of Sclerotic Metastases

Sclerotic metastasis detection uses axial 2D patches because the CT scans have relatively thick slices, then averages random-view ConvNet predictions to reduce false positives.

The dataset contains 532 labeled metastases from 49 patients and 10 control cases, with CT reconstruction slice thicknesses of 2.5–5 mm.
Because coronal and sagittal views have poor diagnostic quality at this slice thickness, spatial transformations are restricted to the axial plane.
False-positive candidate detections provide negative training examples through hard negative mining, while five-fold patient-level cross-validation is used.
Each candidate is represented by 32×32-pixel patches sampled at four physical ROI scales after isotropic CT resampling.
Averaging N predictions produces a per-candidate probability; performance saturates quickly as N increases, with AUC stable at 0.834 for N between 10 and 100.
At 60%, 70%, and 80% sensitivity, false positives per patient decrease from 4 to 1.2, 7 to 3, and 12 to 9.5, respectively.
The two-tiered method markedly improves FROC performance, although training-to-testing differences indicate some overfitting in the 59-patient dataset.

E. Detection of Thoracoabdominal Lymph Nodes

For lymph-node detection, a 2.5D representation combines three orthogonal views, and joint training improves mediastinal performance while maintaining abdominal sensitivity.

The lymph-node dataset contains 176 patients with 388 mediastinal and 595 abdominal lymph nodes labeled as positives.
CT slice thicknesses of ≤1 mm enable a 2.5D representation composed of three orthogonal 2D views for each candidate.
Training uses 6,692 false-positive detections as negative candidates, while labeled lymph nodes are injected to assume 100% candidate-generation sensitivity for training.
The reported improvements over comparison methods are statistically significant, with p < 0.001 in both mediastinal and abdominal evaluations.
Joint training on mediastinal and abdominal candidates raises mediastinal sensitivity from 70% to 77% at 3 FP/vol., while abdominal sensitivity remains stable.
The method achieves 83% sensitivity at 3 FP/vol. in the abdomen, compared with 70.5% at 13.0 FP/vol. in the most recent cited work.

F. 2.5D ConvNets Compared to Shallow Classification

The proposed ConvNet second tier is compared with shallow HoG classification and same-feature random-forest cascades to assess the value of heterogeneous deep representations.

The study compares 2.5D ConvNet aggregation with linear SVMs using HoG features, including pooling and sparse linear decision fusion of 2D scores.
The proposed approach surpasses previous state-of-the-art methods, whereas a two-tier cascade reusing the same image features and random-forest classifiers shows no performance improvement.
These comparisons highlight the importance of heterogeneous image features between candidate generation and candidate classification stages.
The ConvNet approach uses N = 100 random-view observations for the final 3D lymph-node detection comparison.

G. 3D, 2D or 2.5D ConvNets: Alleviating Curse-of-dimensionality via Random View Aggregation

The lymph-node experiments compare 2D, 2.5D, and 3D ConvNets, showing that random-view aggregation and augmentation improve testing performance despite limited training data. In colonic polyp detection, performance saturates after about 10 observations and improves substantially across polyp sizes.

Motivation: Limited 3D training data motivates decomposing volumetric information into 2D or 2.5D views to alleviate the curse of dimensionality.The paper notes that 3D tasks may require at least an order of magnitude more training data than 2D tasks.
Lymph-node comparison: In lymph-node testing, augmented 2D, 2.5D, and 3D ConvNets outperform their corresponding original-data models.Although original-data models train better, augmentation produces stronger testing FROC results.
Colonic-polyp evaluation: At 3 FPs/patient for polyps ≥6 mm, sensitivity rises from 58% with an SVM classifier to 75% with the 2.5D ConvNet.Sensitivity is higher for larger polyps at constant false-positive rates.
Lymph-node comparison: At 3 FPs/patient, augmented 2.5D aggregation outperforms 2D and 3D approaches on the original data.In the original setting, 2D generalizes best, followed by 3D and 2.5D; augmented random-view aggregation reverses this comparison at the stated operating point.
Colonic-polyp evaluation: For ≥10 mm polyps, testing reaches about 95% sensitivity at 1 FP/patient, improving from approximately 65% in the prior system.The paper identifies this operating point as desirable for second-reader use with minimal additional radiologist burden.

I. Limitation & Improvement

The system performs especially well for large colonic polyps, while random 2.5D sampling is less optimal for smaller targets. Local colonic-surface alignment is proposed as a way to guide future sampling.

Performance boundary: The final system shows more appealing performance for large polyps (≥10 mm) than for polyps between 6 and 9 mm.The paper reports consistent FROC improvements for both ≥6 mm and ≥10 mm categories.
Performance boundary: 95% sensitivity at 1 FP/patient is reported for ≥10 mm polyps in testing.The paper presents this as its best reported quantitative benchmark for a large-scale colonic-polyp CADe system.
Limitation and improvement: Random 2.5D view sampling may be suboptimal for small polyps because some views contain only tiny portions of the target.The authors suggest local colonic-surface alignment to guide and constrain sampling.

IV. DISCUSSION AND CONCLUSIONS

The paper extends ConvNets to 2D and 2.5D representations for CT-based CADe, using random observations to improve detection across three pathology categories. The 2.5D design reduces the burden of direct 3D modeling while offering a potential route to classifier-guided visualization.

Discussion and conclusions: ConvNets improve CADe performance for bone lesions, enlarged lymph nodes, and colonic polyps in CT images.The study builds upon existing CADe systems and evaluates random ConvNet observations across three pathology categories.
Discussion and conclusions: Random scales, translations, and rotations can reduce overfitting during training and improve ConvNet classification performance.Testing FROC curves show marked sensitivity improvements at clinically relevant false-positive rates across all three CT datasets.
2.5D representation: 2.5D decomposes each VOI into three-channel images combining orthogonal slices at N reformatted orientations.This representation avoids directly training 3D ConvNets, reducing computational burden and alleviating data-related dimensionality problems.
2.5D representation: 2.5D inputs may permit initialization from ConvNets pretrained on large computer-vision datasets such as ImageNet.The paper identifies this as a potential advantage over Gaussian-random initialization.
Limitations and future work: Larger and deeper ConvNets could improve classification, but insufficient data and high computational and memory costs complicate effective 3D training.These constraints remain relevant when data augmentation is unavailable or inadequate.
Clinical visualization: High-probability 2D views could provide classifier-guided reformatted visualizations at orientations useful for radiologist reading.The paper presents this as a potential extension of its 2D and 2.5D representation.