From Hand-Crafted to Deep Learning-based Cancer Radiomics: Challenges and Opportunities

I. INTRODUCTION

• Diagnostic imaging is noninvasive and can reveal whole-tumor shape, growth, and heterogeneity, unlike focal biopsy, but interpretation is time-consuming and experience-dependent.The growing volume of medical imaging data compounds this interpretation challenge.Source: §I · p. 1
• Radiomics – extracts and analyzes semi-quantitative and quantitative imaging features to build predictive or prognostic models for personalized diagnosis and tumor phenotype assessment.The authors expect it to become an important component of personalized treatment despite challenges to clinical adoption.Source: §I · p. 1
• Aerts et al. analyzed more than 400 CT features from 1,019 lung cancer patients with clinical and gene-expression data to associate radiomics with clinical outcomes.The study showed that radiomics could identify lung and head-and-neck cancers from a single-time-point CT scan.Source: §I · pp. 1–2
• Hand-crafted Radiomics uses a four-stage pipeline, whereas deep learning-based Radiomics may avoid segmented regions of interest and couple feature extraction with analysis.The hand-crafted stages are acquisition/reconstruction, segmentation, feature extraction and quantification, and statistical analysis and model building.Source: §I · p. 2
• The article surveys hand-crafted feature extraction, reduction, and classification; deep architectures and interpretability; hybrid methods; and radiomics challenges and opportunities for signal-processing researchers.It aims to present the field’s state, opportunities, and challenges from a signal-processing perspective.Source: §I · p. 2
• The authors identify gaps in prior reviews, including limited coverage of deep architectures, interpretability, hybrid methods, radiogenomics, and challenges,, –.They argue that signal processing is a core building block of radiomics and that the field merits an integrated introduction for that community.Source: §I · pp. 2–3

Automatic diagnosis

• Esteva et al. tested a deep CNN on 129, 450 clinical images against 21 boardcertificated experts and found performance on par with them.The study suggests automatic diagnosis systems could serve as one of two experts, although such systems remain in their infancy for clinical use.Source: §II · p. 4
• Automated diagnosis systems could improve care quality and reduce medical errors, but privacy risks call for strict regulation of clinical information.Source: §II · p. 4

Cancer diagnosis, tumor detection, and classification

• Radiomics may improve early cancer diagnosis; tumor detection guides targeted treatment, while classification and attribute scoring characterize tumor type, shape, and behavior.Source: §II · p. 7

Survival prediction

• Radiomics can inform survival estimates from image properties and tumor heterogeneity, helping physicians choose treatment and patients make quality-of-life decisions.Source: §II · p. 7

Malignancy and recurrence prediction

• Radiomics offers noninvasive malignancy-likelihood prediction and has shown promising initial results for recurrence prediction, despite limited landmarks after treatment.Source: §II · p. 7

Cancer staging

• Cancer staging informs treatment choice, and Tables II–III summarize applications, HCR, DLR, or hybrid methods, and complementary data sources.Source: Table II · p. 5, Table III · p. 6, §II · p. 7

• Radiomics can guide less invasive biopsies toward suspicious tumor locations and provide complementary diagnostic information, including when a negative biopsy raises the question of re-biopsy.Because tumors are spatially and temporally heterogeneous, biopsy may miss information needed for an inclusive decision; it can also be invasive or risky for some patients.Source: §III · p. 7
• The hand-crafted Radiomics workflow comprises preprocessing, segmentation, feature extraction, feature reduction, and statistical analysis, with the section focusing on the final three steps,,.Preprocessing typically reduces noise and artifacts through smoothing and image enhancement; feature reduction addresses correlated, irrelevant, or overfitting-prone features.Source: §III · pp. 7–8
• Segmentation is critical because hand-crafted features are extracted from segmented tissue, yet manual tumor delineation is time-consuming and observer-sensitive.Conventional methods use intensity, contour-energy optimization, or pixel features for machine-learning classification; deep networks learn discriminative features end-to-end, while ground-truth scarcity makes reproducibility metrics useful,. Reference reported better average overlap similarity for automatic methods than manual delineation.Source: §III · pp. 7–8

Feature categories

• Hand-crafted Radiomics features span first-order intensity and shape [8], second-order texture, and higher-order features, with some workflows initially extracting hundreds.The paper notes that one cited study extracted 400 features before feature reduction.Source: A. Radiomics Feature Extraction · p. 8

Intensity-based Features

• Intensity features summarize ROI pixel values in a histogram; entropy and uniformity are commonly used, but acquisition parameters such as slice thickness and histogram bin selection can affect reliability.Entropy measures intensity randomness, while uniformity estimates consistency; bin choices can also hinder comparisons across studies.Source: A. Radiomics Feature Extraction · p. 8

Shape-based Features

• Shape features quantify 2D or 3D ROI geometry, including volume, surface, sphericity, compactness, diameter, and flatness; they can support malignancy and treatment-response prediction.Sphericity measures roundness; compactness is based on sphericity, so feature selection may exclude one to reduce redundancy.Source: A. Radiomics Feature Extraction · p. 9

Texture-based Features

• Texture features capture spatial relationships between neighboring pixels, making them informative when tissue heterogeneity matters; common matrices include GLCM, GLRLM, NGTDM, and GLZLM,.GLCM models intensity-pair frequencies; GLRLM counts same-intensity runs; NGTDM compares intensities with neighborhood averages; and GLZLM counts connected same-intensity zones.Source: A. Radiomics Feature Extraction · pp. 9–10

Higher Order Radiomics Features

• Higher-order features capture imaging biomarkers across frequencies: wavelets are commonly used, Fourier features can capture gradients, and Minkowski functionals characterize above-threshold pixel patterns.Minkowski functionals use a binarized ROI formed with thresholds across the intensity range; choosing the threshold count may require empirical selection.Source: A. Radiomics Feature Extraction · p. 10

B. Radiomics Feature Reduction Techniques

• Feature reduction targets correlated, irrelevant, or noise-sensitive features; supervised methods favor class discrimination, while unsupervised methods reduce redundancy using PCA, ICA, or zero variance.The section classifies commonly used approaches as supervised or unsupervised.Source: B. Radiomics Feature Reduction… · p. 10
• Selection criteria include reproducibility to preprocessing and annotation, target association, and low correlation among selected features,,.The section gives χ2 testing and Fisher scores as examples for identifying informative features; Fisher scores favor higher variance.Source: B. Radiomics Feature Reduction… · p. 10
• Wilcoxon-test filtering considers each feature’s relation to the class label separately and has shown greater stability and more promising Radiomics predictions.The test is nonparametric and assesses whether two feature sets share the same probability distribution.Source: B. Radiomics Feature Reduction… · p. 10
• Wrapper methods score feature subsets using combined predictive performance, relevancy, and redundancy; greedy forward selection adds features, while backward elimination removes them.Computational difficulty prevents testing every possible subset, and each search stops when another addition or reduction no longer improves performance.Source: B. Radiomics Feature Reduction… · pp. 10–11
• Because they use class labels, supervised methods are susceptible to overfitting and do not readily transfer across applications trained on a given feature set.Filtering ignores feature-to-feature relations, whereas wrappers account for combined subset performance.Source: B. Radiomics Feature Reduction… · pp. 10–11
• Unsupervised reduction removes correlated features without additional information and avoids overfitting, but may not find the optimum feature space; advanced signal-processing methods are an opportunity.The section notes that Radiomics commonly uses simple methods such as PCA and distinguishes linear from nonlinear techniques.Source: B. Radiomics Feature Reduction… · pp. 10–11

Radiomics Statistical Analysis

• Clustering in, explores feature associations with clinical outcomes, while forecasting requires labeled prediction tools such as RF, SVM, and NN.The section also notes that hierarchical clustering is commonly used in Radiomics.Source: C. Radiomics Statistical Analysis · p. 11

Kaplan-Meier Survival Curve (KMS)

• KMS curves, estimate survival over time; feature thresholds separate risk groups, and associates high heterogeneity with shorter survival and compactness with longer survival.KMS curves can be calculated for Radiomics features to compare survival between groups.Source: C. Radiomics Statistical Analysis · p. 12

Cox Proportional Hazards (Regression) Model (PHM)

• Cox PHM models a patient’s hazard using covariates and trained risk coefficients; its conventional linear risk function could be replaced by a nonlinear deep-learning function, not yet investigated in Radiomics.The hazard is the risk of dying at a particular time, and the conventional risk coefficients are trained on historical data.Source: C. Radiomics Statistical Analysis · p. 12

Log-Rank Test

• The Log-Rank Test compares two survival curves non-parametrically, but group sizes can affect results, motivating larger patient samples and equal-sized groups.Source: C. Radiomics Statistical Analysis · p. 12

Evaluation of HCR

• Pipeline performance depends on extraction, reduction, and analysis choices: found feature selection can affect accuracy, whereas, found classification method dominant, with impacts varying by application, dataset, and feature set.Source: C. Radiomics Statistical Analysis · p. 12
• Because FP and FN errors have unequal consequences, accuracy is less informative; ROC area and confusion matrices distinguish error rates, but physician harm-benefit factors remain unquantified in Radiomics.Physicians consider harms and benefits when setting decision thresholds, but these factors are not quantified and used in Radiomics.Source: C. Radiomics Statistical Analysis · pp. 12–13

D. Radiomics Stability

• Test-retest assessment compares features across repeated imaging exams, while inter-observer reliability compares features extracted from multiple observers’ ROI delineations; invariance indicates stability.These approaches probe variation across image sets and segmentations, respectively.Source: D. Radiomics Stability · p. 13
• The intra-class correlation coefficient (ICC) assesses stability in both settings, ranges from 0 (no reliability) to 1 (complete reliability), and is based on ANOVA mean squares.For test-retest, the estimate uses repeated examinations; for inter-observer assessment, it uses residual mean squares and the number of observers.Source: D. Radiomics Stability · p. 13
• The Friedman test is a nonparametric repeated-measures test for inter-observer stability that requires no Gaussian population; features ranked 1 are most stable.It assesses whether distributions differ across multiple observations.Source: D. Radiomics Stability · p. 13
• Features with higher stability are reported to have greater prognostic performance, making stability analysis interpretable as feature reduction.Source: D. Radiomics Stability · p. 13
• For lung CT, Laplacian of Gaussian (LoG), intensity-based, and texture features are more stable, while wavelet and shape-based features are sensitive to segmentation variation.Source: D. Radiomics Stability · p. 13
• Intensity discretization strongly affects texture stability: fixed-sized bins yield more stable and comparable features than a fixed number of bins, though both depend on intensity resolution.Texture analysis therefore requires standardized intensity discretization.Source: D. Radiomics Stability · p. 13
• The Image Biomarker Standardization Initiative (IBSI) provides shared definitions and guidelines spanning acquisition, preprocessing, segmentation, and feature calculation to address reproducibility [73].Source: D. Radiomics Stability · p. 13

E. Radiogenomics

• Radiogenomics links imaging characteristics with gene expression, mutations, and other genomic traits; found 28 radiomics features reconstructed 78% of global gene expression in human liver cancer cells.These associations may help predict treatment response and support clinical decisions without invasive procedures such as biopsy.Source: E. Radiogenomics · p. 13
• Radiogenomics ranks genes by Spearman correlation, then uses GSEA to test whether biologically related gene sets cluster among the strongest positive or negative associations.This gene-set approach addresses the difficulty of biologically interpreting lists focused only on the most positively and negatively correlated genes; results are shown as a heat map of gene-set associations with radiomics features.Source: E. Radiogenomics · p. 14
• When clinical outcomes are unavailable or slow to collect, genomic knowledge can help infer imaging-outcome relationships by linking imaging data to genomics with known clinical associations.In this setting, genomics can fill gaps in knowledge about clinical outcomes.Source: E. Radiogenomics · p. 14
• Lung cancer radiomics supports survival, malignancy, treatment-response, and stage prediction, but automatic tumor segmentation can fail when tumors attach to vessels or walls, leaving segmentation an open problem.Feature relevance varies by task: intensity features correlate with survival, shape features aid malignancy and treatment-response prediction, and texture features are reported as most influential for outcomes.Source: E. Radiogenomics · p. 15
• In 157 NSCLC patients, χ2 selection identified tumor surface area and volume as survival-correlated; smaller tumors appeared necessary, but not sufficient, for long-term survival.The hand-crafted pipeline used expert manual annotations and 11 first-order plus 19 second-order features; tumor location could affect survival but was not represented by ROI-derived features.Source: E. Radiogenomics · p. 15

IV. STATE-OF-THE-ART IN DEEP LEARNING-BASED RADIOMICS

• Deep learning radiomics extracts task-specific features through architectures such as CNNs or auto-encoders, then analyzes them within the network or using an SVM or decision tree.The specified tasks include disease diagnosis, cancer-type prediction, and survival prediction.Source: §IV · p. 16
• Unlike hand-crafted radiomics, deep learning can learn features automatically without prior knowledge and use raw images without ROI segmentation, reducing annotation burden, cost, and observer dependence.The passage also notes that automatic segmentation methods remain error-prone for sensitive decision-making; deep networks can accept combined original, segmented, gradient, or multi-angle images [10],.Source: §IV · p. 16
• Deep-learning radiomics varies by input hierarchy, pre-trained versus scratch-trained models, and network architecture; whole-volume inputs must address variable slice counts, for which RNNs can accommodate variable sizes.Inputs may be individual slices, whole volumes, or patient examinations; model training choices depend on dataset size and available time, and architecture selection is central to extracting practical features.Source: §IV · p. 16

A. Input Hierarchy

• In, patient-level lung-tumor malignancy classification uses the maximum predicted malignancy rate across a patient's CT series, declaring malignancy if any rate exceeds a threshold.A three-layer CNN first extracts DLR from tumor patches for individual CT series.Source: A. Input Hierarchy · p. 16

B. Pre-trained or Raw Models

• Training from scratch can tailor a network to a specific problem, but limited medical datasets risk overfitting and class imbalance can bias predictions toward the negative class.Privacy constraints and the need for expert ground truth limit available datasets; missing cancer cases can be more harmful than falsely labeling healthy patients as cancerous.Source: B. Pre-trained or Raw Models · pp. 16–17
• Deep Radiomics networks can take the original image, segmented ROI, or both, and use extracted features within the network or an external decision model.Source: Figure 3 · p. 17

A. Input Hierarchy

• DLR classification ranges from independent slices to volumes and patients; slice-level predictions alone may not support decisions about an entire VOI.Volume-level methods fuse slice outputs or process all slices for a volume together, while patient-level methods use a series of studies.Source: A. Input Hierarchy · p. 16, Figure 5 · p. 18

B. Pre-trained or Raw Models

• Data augmentation addresses inadequate training data by applying spatial deformations such as rotation or extracting random fixed-size subpatches.Source: B. Pre-trained or Raw Models · p. 18
• Multitask training can address inadequate data and class imbalance by reducing free parameters and sharing a network across tasks.In, a CNN jointly classifies spinal disk grading and narrowing; convolutional layers are shared, tasks branch at the fully connected layers, and their losses are weighted and summed.Source: B. Pre-trained or Raw Models · pp. 18–19
• For DLR extraction, weighting the loss toward the minority class is another strategy for handling class imbalance.Source: B. Pre-trained or Raw Models · p. 19
• Transfer learning followed by fine-tuning trains a network on natural images before retraining it on medical data to address inadequate data and class imbalance,,.Examples include ImageNet-pretrained AlexNet for breast-cancer mammography classification and ImageNet-pretrained CNNs for lung-cancer survival prediction.Source: B. Pre-trained or Raw Models · p. 19

2. Generative Models

• Generative models learn representations from unlabeled data; autoencoder latent codes become radiomic features, and generative-model weights can initialize discriminative models.Autoencoders reconstruct inputs, denoising autoencoders add noise to encourage robust features, and convolutional autoencoders account for spatial correlations; DBNs and DBMs stack RBMs, with DBMs linking all layers undirectionally.Source: C. Deep Learning Architectures in… · p. 19, §1.3 · p. 23

1.1. Convolutional Neural Networks (CNNs)

• CNNs learn convolution kernels rather than using predetermined features; fully connected-layer outputs serve as DLR features for end-to-end prediction or downstream classifiers.Shared weights reduce the number of trainable parameters, and the learned features can distinguish benign from malignant lung tumors when projected into 2D.Source: §1.1 · p. 19
• Multi-view CNNs use seven patches at different scales and achieve higher feature separability than single-scale CNNs, with separability increasing in deeper layers.Multi-scale inputs can capture tumor context and surrounding tissue; multiple CNNs can share parameters and avoid requiring segmented tumor regions.Source: §1.1 · pp. 20–21

1.2. Capsule Networks

• CapsNets use routing by agreement to preserve spatial relationships and improve robustness to transformations such as affine changes and rotations.Unlike CNN pooling, capsule activity vectors represent instantiation parameters, and their lengths represent the probability that an instance is present.Source: §1.2 · pp. 21–22
• CapsNet achieved 86.56% accuracy with 64 filters versus 256 in the original architecture, and outperformed CNNs (78% accuracy for tumor images) on both input types.CapsNet performed better with tumor masks than with original brain images.Source: §1.2 · p. 22

1.3. Recurrent Neural Networks

• LSTMs can process sequential image slices or follow-up examinations; prostate-cancer classification from ultrasound sequences had higher predictive accuracy than decisions from independent images.LSTMs address the vanishing-gradient problem by deciding what information to store or forget.Source: §1.3 · p. 22

Explainability of Deep Learning-based Radiomics

• Expert review of CT-based CNN longevity predictions can validate outcomes but is time-consuming; feature visualization, including final DBN weights, exposes learned features such as curvity.These features may not be clinically distinctive, since clinicians themselves can be unsure about image properties.Source: D. Explainability of Deep… · p. 24
• Sensitivity analysis creates heat maps of influential image regions using pixel effects estimated through CNN backpropagation, but highlighted pixels may not explain the decision and become harder to assess in deeper networks.Source: D. Explainability of Deep… · p. 24
• t-SNE visualization of CNN features for lung-tumor classification produced distinct tumor clusters, indicating discriminating features, but did not reveal the exact reasons for individual decisions.Source: D. Explainability of Deep… · p. 24
• CapsNets reconstruct images from learned class features, and repeatedly tweaking those features for brain-tumor classification suggested decisions may rely on tumor size and deformation.The reconstruction objective is added to classification loss with a smaller weight so it does not distract from extracting discriminative features.Source: D. Explainability of Deep… · p. 24

V. HYBRID SOLUTIONS TO RADIOMICS

• Deep learning and hand-crafted radiomics are on a par for tumor classification and detection, but deep-learning publications are fewer for survival and recurrence prediction, possibly reflecting domain knowledge’s importance.The authors suggest domain knowledge may be less accessible to deep-learning radiomics and more important for survival and recurrence prediction.Source: §V · p. 25, Figure 9 · p. 25
• The authors call for methods combining deep learning’s avoidance of segmentation, feature selection, and human intervention with the domain knowledge available in hand-crafted approaches.Source: §V · p. 25
• When neither approach provides informative features with high predictive capacity, hybrid strategies can combine radiomics with other data sources or combine HCR and DLR features.Source: §V · p. 25

Extracting Radiomics from Different Imaging Modalities

• Combining CT and PET Radiomics improved lung cancer survival-prediction accuracy over either modality alone, while combined MRI Radiomics outperformed single-modal classification for brain tumors.CT lacks the functional and metabolic information available from PET, and different MRI images provide complementary contrast and relaxation information.Source: A. Combination of Radiomics and Other… · pp. 25–26

Integration of Extracted Radiomics with Other Data Sources

• Combining Radiomics with lung cancer staging improved prognostic performance over either alone, and nomograms fuse a Radiomics score with clinical risk factors for survival or disease prediction,,.The score is calculated as a logistic-regression linear combination of selected features, then combined with other influential factors using multivariate logistic regression.Source: A. Combination of Radiomics and Other… · p. 26
• Fusion of Radiomics and genomics classifier outputs improved lung cancer recurrence-prediction accuracy over either classifier alone and more closely resembled the survival Kaplan-Meier plot.The study selected 8 of 35 hand-crafted CT features, trained a Naive Bayesian network, and averaged its output with a classifier trained on two genomics biomarkers.Source: A. Combination of Radiomics and Other… · p. 26

B. Fusion of HCR with DLR (i.e., Engineered Features Coupled with Deep Features)

• Decision-level fusion trains classifiers separately and combines their outputs through soft, hard, or adaptive voting; learns classifier weights with a second-stage classifier for lung cancer classification.Soft voting combines probabilities, hard voting can use majority vote, and adaptive voting learns the importance of each model.Source: B. Fusion of HCR with DLR (i.e.,… · p. 26
• Soft-voting combinations improved breast-cancer prediction accuracy; reports that combined SVMs outperformed individual classifiers on mammogram, ultrasound, and MRI images.In, separate SVMs use hand-crafted and pre-trained-CNN features, with output probabilities averaged for prediction.Source: B. Fusion of HCR with DLR (i.e.,… · p. 26
• Feature-level fusion concatenates deep and hand-crafted vectors, with reporting the highest performance for lung cancer survival prediction and verifying efficiency for lung tumor detection.The cited classifiers for survival prediction were Random Forest and Naive Bayes.Source: B. Fusion of HCR with DLR (i.e.,… · p. 26
• Combining features can cause overfitting when training data are sparse relative to feature count, while reduced dimensions using stability, informativeness, redundancy, and Cox regression for brain cancer survival prediction.Features with small Cox-regression weights were removed as effectless.Source: B. Fusion of HCR with DLR (i.e.,… · p. 27
• For nine lung-tumor attribute scores, combined CNN and DAE features with Haar and HoG features, then used multitask linear regression to account for intertask relations.Nine task-specific CNNs each generated 192 features; the DAE generated 100 features.Source: B. Fusion of HCR with DLR (i.e.,… · p. 27

VI. CHALLENGES, OPEN PROBLEMS, AND OPPORTUNITIES

• The challenges section focuses on limitations specific to Radiomics processing and introduces open problems and signal-processing opportunities.Source: §VI · p. 27

A. Challenges of Hand-Crafted Radiomics

• Most hand-crafted radiomics requires segmented ROIs, but only experts can provide these annotations, making segmentation time- and cost-ineffective.Source: A. Challenges of Hand-Crafted Radiomics · p. 28

B. Challenges of Deep Radiomics

• Deep Radiomics lacks an optimal-architecture strategy, requires sensitivity analysis and medical-image-specific model changes, and needs explainability for clinical use.The relationship between deep features and genetic patterns also remains unestablished.Source: B. Challenges of Deep Radiomics · p. 28
• Deep models need more training data as complexity grows, while cropping can remove size information that matters for medical-image classification.The LIDC dataset contains 244,527 CT images from 1,010 patients, and DBN-misclassified lung tumors were 4% larger than correctly classified ones, possibly because cropping removed size information.Source: B. Challenges of Deep Radiomics · p. 28

C. Open Problems and Signal Processing Opportunities

• Radiomics training data are hard to collect under privacy restrictions, matching clinical characteristics is difficult, and expert-provided ground truth is scarce, motivating weakly or semi-supervised methods.The amount of training data required by deep architectures also grows with the number of trainable parameters.Source: C. Open Problems and Signal Processing… · p. 28, B. Challenges of Deep Radiomics · p. 28
• Scanner and protocol differences produce variation in image contrast and resolution, while expert annotations can differ with specialty, motivating information fusion and unifying comparison schemes.Source: C. Open Problems and Signal Processing… · p. 29
• Because positive disease cases are usually fewer than negative cases, Radiomics classifiers risk majority-class bias, and class imbalance remains unresolved despite weighted metrics.Source: C. Open Problems and Signal Processing… · p. 29
• Breathing-related tumor shifts and scanner limits can destabilize features; super-resolution reconstructs high-resolution images from multiple low-resolution instances, with CNNs showing promising results,,.Increasing the scanner sampling rate can improve resolution at the cost of increased noise.Source: C. Open Problems and Signal Processing… · p. 29
• Radiomics models must handle missing patient data, and immature fusion strategies leave feature-vector combination unresolved despite promising hybrid results.Source: C. Open Problems and Signal Processing… · p. 29

VII. CONCLUSION

• The article integrates hand-crafted, deep, and hybrid Radiomics approaches with practical applications, pipeline modules, and image, clinical, and genomic resources to support further signal-processing research.Source: §VII · p. 29