Lung and Pancreatic Tumor Characterization in the Deep Learning Era: Novel Supervised and Unsupervised Learning Approaches

Sarfaraz Hussein; Pujan Kandel; Candice W. Bolan; Michael B. Wallace; Ulas Bagci

Source-linked AI summary

Lung and Pancreatic Tumor Characterization in the Deep Learning Era: Novel Supervised and Unsupervised Learning Approaches

Sarfaraz Hussein, Pujan Kandel, Candice W. Bolan, Michael B. Wallace, Ulas Bagci

arXiv:1801.03230v3Jan 10, 2018cs.CVcs.AIcs.LGq-bio.QMq-bio.TO

TL;DR

The paper addresses tumor characterization amid limited medical-imaging labels by combining supervised deep learning and unsupervised proportion-SVM approaches. Its graph-regularized MTL approach significantly outperformed comparison methods for lung-nodule risk stratification, while the study also evaluates CAD for IPMN classification.

Problem
Limited labeled medical-imaging examples and variable radiologist labeling make tumor characterization challenging, despite its relevance to early diagnosis and treatment planning.
Method
The paper combines 3D CNN feature extraction, transfer learning, graph-regularized sparse MTL, and clustering-based proportion-SVM for supervised and unsupervised tumor characterization.
Results
The proposed graph-regularized MTL significantly outperformed comparison approaches in lung-nodule classification accuracy and mean score difference.
Takeaways & Limitations
Task-dependent attributes can complement malignancy features, and CAD evaluation for IPMN supports continued exploration of imaging modalities and multimodal architectures.
Takeaways & Limitations
Medical-imaging performance remains limited by domain differences between pretrained natural-image networks and medical images, plus scarce and variable annotations.

Abstract

from arXiv · show

Risk stratification (characterization) of tumors from radiology images can be more accurate and faster with computer-aided diagnosis (CAD) tools. Tumor characterization through such tools can also enable non-invasive cancer staging, prognosis, and foster personalized treatment planning as a part of precision medicine. In this study, we propose both supervised and unsupervised machine learning strategies to improve tumor characterization. Our first approach is based on supervised learning for which we demonstrate significant gains with deep learning algorithms, particularly by utilizing a 3D Convolutional Neural Network and Transfer Learning. Motivated by the radiologists' interpretations of the scans, we then show how to incorporate task dependent feature representations into a CAD system via a graph-regularized sparse Multi-Task Learning (MTL) framework. In the second approach, we explore an unsupervised learning algorithm to address the limited availability of labeled training data, a common problem in medical imaging applications. Inspired by learning from label proportion (LLP) approaches in computer vision, we propose to use proportion-SVM for characterizing tumors. We also seek the answer to the fundamental question about the goodness of "deep features" for unsupervised tumor classification. We evaluate our proposed supervised and unsupervised learning algorithms on two different tumor diagnosis challenges: lung and pancreas with 1018 CT and 171 MRI scans, respectively, and obtain the state-of-the-art sensitivity and specificity results in both problems.

I. INTRODUCTION

The paper develops supervised and unsupervised learning strategies for characterizing lung nodules and pancreatic IPMNs from radiology scans, addressing clinical and labeling challenges. It combines 3D CNNs, graph-regularized sparse multi-task learning, clustering, and proportion-SVM, and evaluates these approaches across CT and MRI settings.

Motivation: Tumor characterization can support earlier diagnosis and treatment planning for lung cancer and pancreatic IPMN, a premalignant condition that can progress to invasive cancer.The paper emphasizes CAD-assisted analysis to improve diagnostic decisions while focusing on clinically consequential lung and pancreatic tumors.
Supervised learning: The supervised approach uses a fine-tuned pre-trained 3D CNN to retain volumetric information from low-dose CT scans for lung-nodule risk stratification.The authors argue that conventional 2D CNNs lose volumetric information and use transfer learning because labeled 3D training examples are limited.
Unsupervised learning: The unsupervised strategy extracts hand-crafted and deep features, clusters samples for initial labels, and progressively refines assignments using proportion-SVM trained on label proportions.This approach targets the limited availability of labeled medical-imaging data and evaluates the suitability of deep features for unsupervised tumor classification.
Supervised learning: Graph-regularized sparse multi-task learning integrates complementary representations of six high-level lung-nodule attributes to improve malignancy prediction.The attributes include calcification, spiculation, sphericity, lobulation, margin, and texture; the framework models them as related tasks.
Evaluation scope: The study evaluates lung-nodule and IPMN characterization using CT and MRI data while investigating different deep-learning features and automatic IPMN diagnosis from MRI.The paper presents the MRI-based IPMN diagnosis task as an area with limited prior automatic-learning research.

A. Problem Formulation

The problem formulation represents radiology-derived features and malignancy scores for lung nodules, then extends supervised prediction to related visual-attribute tasks. The formulation also motivates the use of deep representations and multi-task learning for malignancy characterization.

Data representation: Each lung-nodule sample is represented by a d-dimensional feature vector, while its malignancy score ranges from 1 for benign to 5 for malignant.The input matrix contains n image-derived samples, and the target vector stores their attribute or malignancy scores.
Supervised formulation: The coefficient vector’s sparsity is controlled by parameter t, and the convex formulation is extended with deep-learning and multi-task concepts for benign-versus-malignant characterization.The optimization setting treats the sparse coefficient-learning problem as convex before introducing the broader modeling extensions.

B. 3D Convolution Neural Network (CNN) and Fine-Tuning

The supervised lung-nodule approach uses a fine-tuned 3D CNN to capture volumetric information, then models radiologist-inspired visual attributes jointly with graph and sparsity regularization. It also incorporates disagreement among radiologists as an inconsistency measure.

3D CNN and fine-tuning: A 3D CNN pretrained on Sports-1M is fine-tuned on lung-nodule CT data to obtain dense volumetric features.The network processes fixed 16-slice inputs and supplies 4096-dimensional feature vectors to the MTL framework.
Multi-task learning: Multi-task learning treats lung-nodule visual attributes as distinct tasks and learns a joint model using information shared across them.Each task has parameters W_i, while task dependencies are learned rather than assumed in advance.
Low-rank regularization: Trace-norm regularization encourages low-rank task parameters, but assumes models share a common subspace, which can be restrictive.The trace norm is defined as the sum of the singular values of W.
Graph regularization: Graph regularization represents task relationships through a complete graph whose nodes are tasks and whose edges encode task affinities.The structure matrix is an incidence matrix for the task graph, enabling regularization of deviations between task pairs.
Radiologist disagreement: The objective adds a radiologist-disagreement inconsistency measure to account for differing malignancy scores assigned to the same nodule.The measure uses each expert’s score together with the mean and standard deviation across scores.

D. Optimization

The optimization combines smooth and nonsmooth convex terms, so standard gradient descent is inadequate for obtaining sparse solutions. The method instead uses an accelerated proximal-gradient procedure.

Optimization: Standard gradient descent fails because the ℓ1-norm is nondifferentiable at W = 0 and does not provide sparse solutions.The objective’s nonsmooth ℓ1 component is replaced by estimates using a proximal operator.
Optimization: Accelerated proximal gradient solves the optimization with a convergence rate of O(1/m^2), where m controls the iteration count.This method is applied because the objective contains both smooth and nonsmooth convex parts.

IV. UNSUPERVISED LEARNING METHODS

The unsupervised framework addresses the cost and scarcity of medical-image annotations through a three-step pipeline: clustering, label-proportion estimation, and tumor categorization.

Unsupervised learning: Clustering appearance features first produces an initial set of tumor labels without requiring manual annotation.The initial labels are estimated from image-derived features.
Unsupervised learning: Label proportions are then computed for each cluster using the initial assignments.These proportions provide group-level supervision for the subsequent learning stage.
Unsupervised learning: The framework finally uses cluster assignments and label proportions to learn tumor categorization.The three-step design is motivated by laborious, expensive, and time-consuming medical-image annotation.

A. Initial Label Estimation

Initial labels are estimated by clustering image features with k-means, after which cluster assignments are converted into label proportions for proportion-SVM learning. Because these labels may be noisy, the subsequent model treats instance labels as latent and uses group-level proportions.

Initial Label Estimation: k-means partitions n image-feature vectors into 2 ≤ k < n clusters using an assignment matrix A.The clustering objective minimizes distances between samples and their assigned cluster means μ_v.
Initial Label Estimation: The assignment matrix estimates labels that are used to compute label proportions for the clustered data.Clustering is only an initial step; assignments are progressively refined by proportion-SVM.
Learning with the Estimated Labels: Noisy initial instance labels motivate modeling labels as latent variables and using group-level labels instead.Clusters are treated as disjoint groups whose majority samples correspond to a class, with known group-level proportions guiding learning.
Initial Label Estimation: The unsupervised approach uses GIST features, k-means clustering, cluster proportions, and proportion-SVM in sequence.The resulting pipeline learns a discriminative model from features and label proportions.
Learning with the Estimated Labels: Proportion-SVM minimizes hinge loss under constraints requiring estimated label proportions to remain within ε of the true proportions.The formulation uses instance-level labels, input features, a cost parameter, and maximum-margin classification.

C. Calculating Label Proportions

The unsupervised framework estimates group-level label proportions from clustered training samples and uses them with image features to train proportion-SVM, while reserving ground-truth labels for evaluation.

The method assumes most instances in each group share a label, with a small number of possible outliers.Label proportions act as soft labels for groups.
The proportion-SVM is trained using image features and estimated label proportions to classify testing data.
Ground-truth benign/malignant labels are used only for evaluation, not for estimating proportions or training proportion-SVM.Clustering and proportion calculation use training data, while testing data remains unseen during proportion-SVM training.
The framework fixes the number of clusters at 2, corresponding to benign and malignant classes.

V. EXPERIMENTS

Experiments evaluate lung nodule characterization on LIDC-IDRI CT scans and IPMN classification on T2 MRI scans, using radiologist annotations and preprocessing for pancreatic images.

The lung evaluation uses 1018 LIDC-IDRI CT scans with slice thicknesses ranging from 0.45 mm to 5.0 mm.At most four radiologists annotated nodules at least 3.0 mm in diameter.
Lung evaluations include 1340 nodules interpreted by at least three radiologists.Mean scores were used across radiologists for malignancy and six nodule attributes.
The IPMN dataset contains T2 MRI axial scans from 171 subjects, including 38 normal and 133 IPMN cases.The scans were labeled by a radiologist as normal or IPMN.
Pancreatic MRI preprocessing applies N4 bias-field correction followed by curvature anisotropic filtering.The filtering smooths images while preserving edges.

C. Evaluation and Results- Supervised Learning

The supervised approach fine-tunes a pretrained 3D CNN for malignancy and attribute tasks, then applies graph-regularized multi-task learning, which outperforms comparison methods on lung nodule characterization.

A 3D CNN pretrained on Sports-1M is fine-tuned with binary labels for malignancy and six nodule attributes across seven tasks.The study uses the score midpoint as the pivot and performs 10-fold cross-validation.
The final CNN feature representation is taken from the first fully connected layer and contains 4096 dimensions.
The graph structure matrix encodes correlations between tasks estimated from a normalized coefficient matrix obtained with least squares and lasso.The correlation matrix is thresholded to obtain a binary graph structure matrix.
15% and 11% classification-accuracy gains are obtained over GIST and trace-norm MTL, respectively.The proposed graph-regularized MTL also improves classification accuracy by 5% over the pretrained network.
32% and 27% reductions in average absolute score difference are obtained versus GIST and trace-norm MTL, respectively.

D. Evaluations and Results- Unsupervised Learning

The unsupervised evaluation combines clustering with proportion-SVM for lung nodules and IPMN, and examines whether deep features support clustering-based classification. The framework improves accuracy and sensitivity, while ReLU-enhanced features are generally more discriminative.

Unsupervised evaluation and results: The unsupervised framework uses GIST image features, two clusters for benign and malignant classes, and 10-fold cross-validation.Cluster-derived training label proportions are input to a linear-kernel proportion-SVM.
Unsupervised evaluation and results: Almost 21% and around 7% sensitivity improvements over clustering+SVM are obtained for lung nodules and IPMN, respectively.
Unsupervised evaluation and results: Around 24% and 9% accuracy improvements over clustering are obtained for lung nodules and IPMN, respectively.
Deep features for unsupervised classification: ReLU features are more discriminative than corresponding features without ReLU for clustering-based lung nodule and IPMN classification.The strongest reported accuracy uses VGG-fc7-ReLU for IPMN and VGG-fc8-ReLU for lung nodules.
Deep features for unsupervised classification: GIST features achieve performance comparable to deep features in unsupervised classification.The authors relate this to ImageNet-trained filters being tuned more to natural-image than medical-image variations.
Supervised comparison: For smaller IPMN datasets, deep features perform better than GIST in supervised classification.The supervised comparison uses ground-truth labels and applies adaptive synthetic sampling to balance classes.

VI. DISCUSSION AND CONCLUDING REMARKS

The paper combines supervised and unsupervised strategies for tumor characterization, using 3D CNN features, graph-regularized sparse MTL, clustering, and proportion-SVM. The discussion highlights complementary imaging attributes, limitations of transferred features, and unresolved challenges in pancreatic segmentation and medical-image labeling.

Supervised learning: The lung-nodule framework combines 3D CNN feature extraction with graph-regularized sparse MTL for malignancy determination.The study presents this as an early evaluation of MTL and transfer learning with 3D deep networks for lung-nodule risk stratification.
Supervised learning: 3D CNN features associated with spiculation, texture, calcification, and other imaging attributes complement malignancy-related features within graph-regularized sparse MTL.The framework integrates these learned representations rather than manually modeling the attributes.
Unsupervised learning: Clustering followed by progressively refined proportion-SVM labels produced promising unsupervised malignancy-classification results and outperformed other evaluated methods.This approach addresses the cumbersome and expensive acquisition of radiologist annotations in medical imaging.
Unsupervised learning: GIST features achieved comparable results to pre-trained network features for some lung-nodule classification instances.The paper attributes this observation to ImageNet-trained filters being tuned more to natural-image nuances than medical images.
Limitations and future directions: Pancreatic MRI analysis remains constrained by the difficulty of segmenting the pancreas, while scarce labels and radiologist variability remain broader medical-imaging challenges.The discussion points to other modalities, joint segmentation and diagnosis, and weakly supervised or active-learning approaches as future directions.