Source-linked AI summary
Radiomics-Conditioned Modulation of RenalCLIP Features for Clear Cell Renal Cell Carcinoma Classification
Yuan Liang, Sourav Bhattacharjee, Abraham Campbell
TL;DR
Distinguishing ccRCC from renal tumours with overlapping CT appearances is challenging, and radiomics may complement learned image representations when labelled data are limited. This paper uses radiomics to modulate RenalCLIP features while retaining a direct radiomics contribution; residual FiLM achieved the highest mean AUC among evaluated RenalCLIP fusion strategies in both cohorts.
Problem
Renal masses can have overlapping imaging appearances, and conventional measurements capture only part of their spatial variation; radiomics may complement learned representations when labelled data are limited.
Source: §1 · pp. 1–2Method
The proposed residual FiLM framework uses radiomics to modulate RenalCLIP features and to provide a direct residual representation for the classifier.
Source: §1 · p. 2, §2.3 · pp. 3–4Results
AUCs of 0.804 internally and 0.854 externally; residual FiLM had the highest mean AUC among evaluated RenalCLIP fusion strategies in both cohorts.
Source: §4 · p. 9Takeaways & Limitations
The findings support conditional modulation for integrating radiomics and RenalCLIP representations and motivate confirmation across larger, independent cohorts.
Source: §4 · p. 9The study is retrospective with a relatively small internal test cohort, and comparisons between frozen-feature and end-to-end approaches warrant caution because preprocessing and optimisation differed.
Source: §3.6 · p. 9
Abstract
from arXiv · showhide
Radiomics provides quantitative descriptions of tumour appearance that may complement disease-specific foundation models in small labelled cohorts. We investigate this complementarity for computed tomography-based classification of clear cell renal cell carcinoma. Our framework uses radiomics to modulate RenalCLIP features through feature-wise linear modulation (FiLM), while retaining a direct radiomics contribution. Internal testing and external validation compare it with conventional fusion strategies and reference classifiers. The FiLM model achieves an area under the receiver operating characteristic curve (AUC) of 0.804 internally and 0.854 externally, with the highest mean AUC among the evaluated RenalCLIP fusion strategies in both cohorts. Pathway ablations examine the contributions of conditional modulation and the direct radiomics residual, while feature permutation highlights the role of tumour texture. These findings support radiomics as a useful complement to RenalCLIP in a small labelled cohort and identify FiLM as an effective approach to integrating their representations for robust renal tumour classification.
1 Introduction
Renal tumour classification must distinguish lesions with overlapping appearances, while conventional measurements capture only part of their spatial variation. The section motivates radiomics as a compact complement to RenalCLIP and introduces a radiomics-conditioned residual FiLM framework for ccRCC classification.
1 Introduction
- Radiomics descriptors quantify tumour shape, intensity, and texture, linking classifier inputs to measurable properties that can be difficult to assess consistently by visual inspection.They summarise three-dimensional geometry, intensity distributions, and spatial relationships between grey levels.Source: §1 · pp. 1–2
- Radiomics can offer a compact complement to learned image representations when labelled data are limited, although predefined descriptors cannot capture every aspect of tumour appearance.Its feature extractor does not require target-cohort training, and feature selection can prioritise reproducible, nonredundant measurements.Source: §1 · p. 2
- RenalCLIP uses disease-specific visual–textual pretraining, but adaptation remains relevant for a particular histological distinction; the authors hypothesise radiomics can supplement it with few labelled examples.The section also describes residual networks and vision–language pretraining as alternative ways to represent images.Source: §1 · p. 2
- The proposed residual FiLM framework uses radiomics both to modulate RenalCLIP features and to contribute directly to the classifier.It is compared with concatenation, alternative gates, globally shared affine modulation, and conventional radiomics and convolutional references, with internal and external evaluation, pathway ablations, and feature permutation analysis.Source: §1 · p. 2
2 Methods
The study evaluates ccRCC classification on an internal KiTS23 cohort and a deduplicated external cohort combining RCC-AID and TCGA cases. The internal data use a fixed patient-level split, and all external methods use the same selected scan and mask per patient.
2.1 Study Cohorts
- The internal cohort included 396 KiTS23 patients, split by patient into 237 training, 99 validation, and 60 test cases, with an approximately 70% ccRCC prevalence in each subset.ccRCC was the positive class and other histologies the negative class; available tumour delineations were combined using STAPLE consensus.Source: §2.1 · p. 2
- The external cohort contained 111 patients after deduplication and image-quality review, including 85 ccRCC and 26 nonccRCC cases; all methods used the same scan and tumour mask per patient.The cohort combined RCC-AID data with previously assembled TCGA cases using AIMI annotations.Source: §2.1 · p. 3
Radiomics.
The methods standardise CT volumes and extract radiomics descriptors, then select a reproducible, less redundant subset. A fixed RenalCLIP encoder supplies tumour-weighted image features for fusion.
Radiomics.
- CT volumes were oriented, resampled to 1-mm isotropic spacing, and clipped to [−150, 200] HU before PyRadiomics extraction using IBSI-described definitions.The descriptors combine original-image shape with first-order and texture features from original and LoG images; LoG scales were σ ∈{1, 2, 3} mm and bin width was 25 HU.Source: Radiomics · p. 3
- Feature selection reduced 386 initial descriptors to 73 using inter-annotator ICC, MAD, and correlation filters, retaining the highest-MAD feature per cluster.Median imputation and standardisation were fitted on training patients and applied unchanged to subsequent datasets.Source: Radiomics · p. 3
- A fixed RenalCLIP 3D ResNet18 encoder produced a 256-dimensional, L2-normalised image vector by tumour-mask-weighted pooling of its third residual-stage feature map.Its pretraining data came from independent institutional cohorts in China that did not overlap with the study datasets.Source: RenalCLIP · p. 3
RenalCLIP.
The CT input was centred on the tumour and resampled and cropped to a fixed field of view. Windowing mapped intensities to [0,1], with the same processing across cohorts.
RenalCLIP.
- The input field of view was centred on the tumour centroid in the axial slice with the largest tumour area.The initial 87.5 × 87.5 × 100 mm field was sampled to 140 × 140 × 32 voxels, then centre-cropped to a final 80 × 80 × 100 mm field.Source: RenalCLIP · p. 3
- Trilinear sampling and centre cropping produced a final 80 × 80 × 100 mm field of view from the tumour-centred volume.The sampling grid was 140 × 140 × 32 voxels before the 128 × 128 × 32 centre crop.Source: RenalCLIP · p. 3
- Windowing at 50 HU centre and 500 HU width mapped intensities to [0,1], using the same processing across cohorts.Source: RenalCLIP · p. 3
2.3 Radiomics-Conditioned Residual Modulation
The framework uses radiomics to modulate image features while preserving a direct radiomics contribution to the fused representation. The image encoder is fixed, while the projections, conditioning network, and classifier are learned.
2.3 Radiomics-Conditioned Residual Modulation
- Radiomics conditions projected image features through an affine transformation while also contributing directly to the fused representation.The multiplicative term adapts image-coordinate contributions within a bounded range, the additive term adjusts offsets, and the direct residual preserves projected radiomics information.Source: §2.3 · pp. 3–4
- The image and radiomics inputs are projected into a common feature space, and a 128 →128 →256 conditioning network generates patient-specific modulation parameters.Source: §2.3 · pp. 3–4
- The image encoder remains fixed, while the projections, conditioning network, and classifier are learned.Source: Figure 1 · p. 4
2.4 Comparison Methods
The comparison methods include conventional radiomics and convolutional references alongside RenalCLIP classifiers using concatenation, gates, or global affine modulation.
2.4 Comparison Methods
- The conventional references are a radiomics-based radial-basis-function SVC and a 3D ResNet18 trained from scratch, with radiomics-gated and radiomics-FiLM variants.The convolutional models use normalised 128^3 tumour-centred CT inputs and 512-dimensional ResNet18 representations for fusion.Source: §2.4 · p. 4
- RenalCLIP comparisons include an image-feature MLP and concatenation of its 256-dimensional representation with 73 radiomics features for a 329-dimensional classifier input.Source: §2.4 · p. 4
- Gate generators condition on radiomics alone or jointly on image and radiomics, while global affine modulation shares γ and β across patients.The global affine comparison tests adaptive conditioning within the same residual formulation.Source: §2.4 · p. 5
2.5 Training and Evaluation
RenalCLIP-based classifiers are trained with AdamW and selected by internal validation AUC; evaluation uses AUC and AP across internal and external cohorts with different uncertainty estimates for repeated-run and single-model methods.
2.5 Training and Evaluation
- RenalCLIP classifiers use cross-entropy and AdamW at learning rate 10−3, while convolutional references use class-weighted cross-entropy and Adam at 10−4 with mixed-precision training.Models are selected by internal validation AUC and applied to test and external cohorts without further fitting.Source: §2.5 · p. 5
- AUC is the primary metric and AP is complementary; RenalCLIP methods use three independent runs, whereas SVC and convolutional references each use one model.Source: §2.5 · p. 5
- Internal repeated-run results report mean and sample SD, and single-model references report 95% CIs; external results use patient-bootstrap CIs from 2,000 resamples.The bootstrap preserves class counts and prediction correspondence across runs; CIs quantify patient-sampling uncertainty for fitted models.Source: §2.5 · p. 5
2.6 Pathway Ablations and Feature Permutation
Inference-time pathway ablations isolate the direct radiomics residual and conditional modulation, while permutation analysis estimates descriptor importance from AUC decreases.
2.6 Pathway Ablations and Feature Permutation
- Two inference-time ablations remove the direct radiomics residual or disable modulation while preserving the other pathway, with all fitted weights held fixed.The no-direct variant retains conditional modulation; the no-modulation variant retains the projected representations.Source: §2.6 · p. 5
- Each radiomics descriptor is permuted across internal-test patients 50 times, and importance is the AUC decrease averaged across permutations and then runs.The same permutations are used across runs, and radiomics projection and modulation parameters are recomputed after each permutation.Source: §2.6 · p. 5
3 Results and Discussion
Radiomics-fused RenalCLIP models achieved their strongest mean AUCs with FiLM in both cohorts, while ablations and permutation analyses examined the model’s information pathways and feature reliance. The study also notes limits on interpreting ablations, permutation effects, and comparisons across evaluation settings.
3.1 Internal Classification Performance
- 0.804 internal AUC was the highest mean among evaluated RenalCLIP fusion strategies for FiLM, improving on RenalCLIP alone by approximately 9.8 percentage points.Global affine modulation ranked next, followed by gated fusion and concatenation.Source: §3.1 · pp. 5–6, Table 1 · p. 6
3.2 External Evaluation
- 0.854 external AUC was FiLM’s highest mean among evaluated RenalCLIP fusion strategies, although SVC retained a slightly higher AUC.FiLM had a higher AP than SVC; margins over other fusion strategies were small.Source: §3.2 · p. 6, Table 2 · p. 7
3.3 Pathway Ablations
- Removing the direct radiomics residual and disabling conditional modulation reduced internal mean AUC by 0.79 and 0.22 percentage points, respectively; these fitted-model pathways overlap.Retaining modulation lets radiomics influence predictions indirectly, while the residual preserves its explicit contribution; the ablations do not establish either pathway as individually indispensable.Source: §3.3 · p. 7
3.4 Radiomics Feature Importance
- Texture descriptors dominated the highest-ranked permutation effects, led by LoG 2-mm GLCM information measure of correlation 2, original-image GLDM dependence entropy, and LoG 1-mm zone entropy.Because permutation affects both radiomics projection and modulation parameters, and correlated descriptors may share information, these scores indicate model reliance rather than direct biological attribution.Source: §3.4 · p. 7, Figure 2 · p. 8
3.5 Complementarity and Fusion Design
- Radiomics supplies explicit predictive information and patient-specific context for adapting RenalCLIP features, while its incremental benefit from conditional modulation is modest.The direct residual preserves descriptors useful for classification; simpler fusion interactions also exploit radiomics effectively.Source: §3.5 · pp. 8–9
3.6 Limitations and Future Work
- The retrospective study’s internal test cohort was relatively small, and comparisons between frozen-feature and end-to-end methods require caution because preprocessing and optimisation differed.The authors call for larger independent cohorts and repeated training of all reference models.Source: §3.6 · p. 9
4 Conclusion
Residual FiLM combines radiomics-conditioned modulation of RenalCLIP features with a direct radiomics contribution. It achieved the highest mean AUC among evaluated RenalCLIP fusion strategies in both cohorts, and the analyses highlighted overlapping radiomics contributions and texture descriptors.
4 Conclusion
- Residual FiLM achieved AUCs of 0.804 internally and 0.854 externally, with the highest mean AUC among evaluated RenalCLIP fusion strategies in both cohorts.The framework uses radiomics to modulate image features while preserving their direct contribution.Source: §4 · p. 9
- Pathway ablations indicated overlapping radiomics contributions, while permutation analysis highlighted texture descriptors.Source: §4 · p. 9
- The findings support conditional modulation for integrating radiomics and RenalCLIP representations and motivate confirmation across larger, independent cohorts.Source: §4 · p. 9