Fusion techniques of time frequency-based images to predict the outcome of rTMS depression therapy

• The title identifies fusion techniques of time-frequency-based images for outcome prediction.Source: Highlights · p. 2

rTMS Depression Therapy

• Blend and montage fuse time-frequency images for depression-therapy outcome prediction using a lightweight custom CNN.The images are generated from EEG signals.Source: rTMS Depression Therapy · p. 2
• The study compares fused time-frequency techniques against traditional time-frequency methods.Source: rTMS Depression Therapy · p. 2
• The reported gap between experimental setups is the main result; segment-level validation raises accuracy from chance to near-perfect on this data.The paper cautions that similar EEG-literature figures should be read alongside fold construction.Source: rTMS Depression Therapy · p. 4

1. Introduction

• Selecting depression therapy remains a trial-and-error process, motivating treatment-response prediction from EEG and machine learning.Source: §1 · p. 5
• The authors present what they describe as the first subject-disjoint evaluation of EEG-based rTMS response prediction, addressing earlier reports that did not specify fold construction.They characterize this as a properly validated estimate when patients do not appear in both training and testing.Source: §1 · p. 6
• All eight architectures move from patient-level AUC 1.000 to chance when fold construction changes, with training settings otherwise held constant.The same code and training settings were used across both protocols.Source: §1 · p. 6
• The study generates EEG time-frequency images using CWT, DWT, ST, and SSWE, then applies blending or montage fusion with a custom CNN.The proposed CNN is described as lightweight and computationally efficient.Source: §1 · p. 6
• The comparison includes thirteen alternative architectures spanning pretrained vision models, EEG-specific models, a CNN-LSTM hybrid, and classical classifiers.Source: §1 · p. 7
• The evaluation reports patient-level bootstrap confidence intervals, Brier scores for calibration, and patient-level permutation testing.The paper notes these evaluation practices are not standard in the literature.Source: §1 · p. 7

2.1. Feature Fusion

• The proposed method outperformed existing techniques for MDD diagnosis, achieving 88.73% accuracy; Zhang et al. fused spatial-temporal EEG features and reported 96.33% performance.Source: §2.1 · p. 7

2.2. Machine Learning

• Prior rTMS prediction studies applied classifiers such as SVM, KNN, and Elastic Net to EEG or combined EEG and clinical features, reporting varied accuracy.Examples include SVM accuracy of 94.31% using GA-selected features and 97.33% using PLV plus clinical features.Source: §2.2 · pp. 8–9

2.3. Convolutional Neural Network

• CNN studies transformed EEG asymmetry or STFT signals into images for depression classification, while DeprNet’s accuracy differed between recordwise and subjectwise splits.DeprNet reported 99.37% accuracy with recordwise splitting and 91.4% with subjectwise splitting.Source: §2.3 · p. 10
• CNN studies reported 97.27% accuracy for a brain-network model and 89.19% average accuracy on dataset 2b plus 97.7% for subject 7 on dataset III.Source: §2.3 · pp. 10–11

2.4. Transfer Learning (TL)

• For rTMS response classification, reported 92.28% accuracy for a DEO ensemble, while reported 97.1% with frontal TL-LSTM-Attention based on EfficientNetB0.Source: §2.4 · p. 11

2.5. Artificial Neural Network (ANN)

• ANN studies of rTMS responders reported 89.12% accuracy in a 147-patient study and 89.09% accuracy in a 55-patient study.Source: §2.5 · p. 11

2.6. Hybrid Architecture

• Five Convolutional LSTM models for rTMS response prediction in 50 patients reported test accuracies of 93.3%, 98%, 95.2%, 99.2%, and 96.6%.Source: §2.6 · p. 12
• A raw-EEG hybrid combining pretrained CNNs and BLSTM cells with weighted-majority voting achieved 98.51% accuracy for rTMS response prediction.Source: §2.6 · p. 12

3. Materials and Methods

• The experiments used an NVIDIA H100 GPU to accelerate time-frequency fusion and deep-learning processing.Source: §3 · p. 12

3.1. Datasets and Clinical Characteristics

• The proprietary hospital database contains independent cohorts of 15 and 46 patients, with MDD diagnoses established by clinical professionals using standard criteria.The primary and secondary cohorts were recorded during different periods.Source: §3.1 · p. 12
• Response was defined as at least a 50% BDI-II reduction after four weeks; the primary cohort included 9 responders and 6 non-responders, and the secondary included 23 of each.A mental health specialist verified outcomes.Source: §3.1 · p. 13

3.2. rTMS Treatment Protocol and EEG Preprocessing

• Patients received 20 rTMS sessions over four weeks, with stimulation delivered to the left DLPFC at 10 Hz and 110% of resting motor threshold.Each session consisted of 3,000 pulses.Source: §3.2 · p. 14
• Resting EEG was recorded before treatment from 19 electrodes at 500 Hz, filtered and artifact-cleaned, then divided into non-overlapping 20-second segments.All segments from each patient remained in one fold; normalization was performed within each fold using training data only to prevent leakage.Source: §3.2 · p. 14

3.3. Time-frequency method fusion

• Montage places two time-frequency images side by side, while blending overlays them with alpha blending to support richer feature extraction.The study applies both fusion mechanisms to CWT paired with DWT, ST, or SSWE.Source: §3.3 · p. 15

3.3.1. Fused CWT and ST

• CWT and S-Transform images are generated from each 20-second EEG segment, then combined using montage or blending to form CWT_ST image databases.CWT captures localized time-frequency features, while ST represents temporal signal variations.Source: §3.3.1 · p. 16

3.3.2. Fused CWT and DWT

• DWT decomposes signals at discrete scales into frequency bands using high- and low-pass filters, helping detect sudden variations and transient components.Source: §3.3.2 · p. 17
• EEG segments were transformed with DWT into scalogram images and fused with CWT frames using montage or blending.The DWT processing used 20-second segments sampled at 500 Hz.Source: §3.3.2 · p. 17

3.3.3. Fused CWT and SSWE

• CWT and SSWE time-frequency images were fused using montage or blending after 20-second segmentation and SSWE transformation.Signals were normalized before SSWE was applied to each segment.Source: §3.3.3 · p. 18

3.4. Data Splitting and Cross-Validation Setup

• The study used segment-level 10-fold cross-validation, which allowed a patient’s segments to appear in both training and testing, and separate stratified group folds that kept patients disjoint.The segment-level setup was used in Sections 4.1–4.4, while subject-level experiments used the separate protocol.Source: §3.4 · p. 19

3.5. Proposed CNN

• The proposed custom CNN performs binary classification on 224 × 224 × 3 images, using three convolutional layers with 8, 16, and 32 filters, pooling, 25% dropout, and a single-neuron sigmoid output.Source: §3.5 · p. 19

3.6. TL Architecture

• VGG16, Xception, ResNet152V2, MobileNetV2, and DenseNet201 transfer-learning models shared a classification head with global average pooling, a 256-unit ReLU layer, dropout, and sigmoid output.The models used ImageNet-pretrained bases for responder versus non-responder classification.Source: §3.6 · pp. 19–20

3.7. Performance Evaluation

• Evaluation used confusion-matrix accuracy, precision, recall/sensitivity, specificity, and F1, plus ROC-AUC and Brier score for probability calibration.A base-rate predictor scores 0.24 on the primary cohort (9/15 responders) and 0.25 on the secondary cohort (23/46 responders); lower Brier scores are better.Source: §3.7 · pp. 20–21
• Subject-level experiments are repeated across randomized initialization seeds, with performance reported as mean ± standard deviation.The evaluation also reports how many configurations were run and includes tests that do not reach significance.Source: §3.7 · pp. 21–22
• AUC 95% confidence intervals use 2,000 bootstrap resamples of patients with replacement, reflecting that segments within a patient are not independent.The interval uses the 2.5th and 97.5th percentiles of the bootstrap distribution.Source: §3.7 · p. 21
• Patient-level permutation tests shuffle labels across whole patients and rerun the complete cross-validation pipeline to assess performance under a no-information null.The study uses p < 0.05, reports Bonferroni-corrected thresholds when multiple configurations are tested on a cohort, and reports all test outcomes.Source: §3.7 · pp. 21–22

4.1. Preliminary experiment

• The experiment evaluates CWT, DWT, ST, and SSWE with the proposed CNN across responder and non-responder classes.Table 3 summarizes the classification metrics for the primary dataset.Source: §4.1 · p. 22, Table 3 · p. 23
• 98.67% accuracy and 98.52% AUC are reported for CWT, the highest-performing time-frequency method with the proposed CNN.The comparison covers responder and non-responder classes on the primary dataset.Source: §4.1 · p. 22, Table 3 · p. 23
• CWT images are selected for further fusion with other time-frequency methods.The next experiments use CWT as the method to combine with other representations.Source: §4.1 · p. 22

4.2. Pretrained architectures

• VGG16, Xception, ResNet152V2, MobileNetV2, and DenseNet201 are evaluated using CWT time-frequency images.The models are described as state-of-the-art pretrained models.Source: §4.2 · p. 23
• The pretrained-model comparison is intended to assess their performance on CWT time-frequency images.The authors also use it as a further comparison.Source: §4.2 · p. 23
• Table 4 summarizes the results of the pretrained-model experiment.The table is titled as a performance comparison of the CWT method with the proposed CNN versus pretrained models.Source: §4.2 · p. 23

4.3. Fused time frequency methods

• Montage places two time-frequency images side by side, while blending overlays them; the three pairs are CWT_ST, CWT_SSWE, and CWT_DWT.Together, the two fusion techniques produce six fused image datasets evaluated with the same proposed CNN architecture.Source: §4.3 · pp. 23–24
• 99.90% montage accuracy and 98.33% blending accuracy are reported for CWT_ST, the highest-accuracy fused method with the proposed CNN.Table 5 presents the fused-method performance comparison.Source: Table 5 · p. 24, §4.3 · p. 24
• 99.90% accuracy and 99.91% AUC are reported for the proposed CNN on montage CWT_ST, the best overall performance in the comparison with pretrained models.The compared pretrained models include VGG16, Xception, ResNet152V2, MobileNetV2, and DenseNet201.Source: Table 6 · p. 24, §4.3 · pp. 24–25
• Accuracy rises from 98.67% to 99.90% for the proposed CNN when comparing individual CWT with fused CWT_ST.The text also reports increases for VGG16 (83.94% to 97.71%), Xception (87.07% to 94.40%), and ResNet152V2 (90.78% to 98.96%).Source: §4.3 · p. 25

4.4. Testing with Secondary rTMS database

• 98.67% accuracy made CWT the highest-performing individual time-frequency method in the comparison.The authors then applied montage and blending to combine time-frequency methods.Source: §4.4 · p. 25
• CWT_ST and CWT_SSWE combinations achieved higher accuracy than individual time-frequency methods in both montage and blending configurations.The authors report that fusion led to better classification performance on the secondary dataset.Source: Table 7 · p. 26, §4.4 · p. 26
• Montage fusion produced F1-scores of 92% for both non-responsive and responsive cases, and the authors report 14–18% absolute accuracy gains over pretrained models.They describe consistent results across fusion techniques as indicating method stability.Source: §4.4 · pp. 27–28
• 91.90% accuracy with montage fusion and 91.43% with blending; the proposed CNN outperformed pretrained models on the secondary dataset.Montage accuracy was 91.90% versus 73.86% for VGG16, 75.42% for ResNet152V2, and 77.74% for MobileNetV2; blending reached 91.43% versus 75.04–76.47% for pretrained networks.Source: §4.4 · p. 27, Table 8 · p. 28

4.5.1. Subject-Disjoint Cross-Validation Protocol

• Subject-disjoint validation keeps each patient’s segments in one fold; the primary cohort uses stratified group three-fold CV and the secondary cohort stratified group five-fold CV.Out-of-sample predictions are pooled across folds, each patient contributes one prediction, and confidence intervals use patient-level bootstrap resampling.Source: §4.5.1 · pp. 28–29

4.5.2. Testing with Primary rTMS database

• On the primary cohort, standalone ST has AUC 0.485 ± 0.079 and CWT 0.478 ± 0.072; every method’s AUC confidence interval contains 0.5.The intervals are wide, spanning roughly 0.14 to 0.81.Source: §4.5.2 · pp. 29, 35, Table 9 · p. 30
• Montage CWT_ST reaches AUC 0.874 ± 0.183 and 82.7% accuracy, the highest subject-level performance reported.Its confidence interval is 0.718 to 0.975.Source: §4.5.2 · p. 30, Table 10 · p. 31
• Five of six primary-cohort fusion pairings are at chance; Blend CWT_ST reaches 0.474 ± 0.098, below CWT alone at 0.478 and ST alone at 0.485.Montage keeps the two representations side by side, whereas blend averages them into one image; the same montage strategy outperforms blend on the same pair of methods.Source: §4.5.2 · p. 30, Table 10 · p. 31
• The proposed CNN is the smallest of six architectures above AUC 0.82; SleepEEGNet, CNN-LSTM, MobileNetV2, VGG16, and DenseNet201 have 1.9, 2.4, 6.4, 36, and 46 times as many parameters.The proposed model reaches the same 82.7% accuracy as CNN-LSTM and DenseNet201.Source: Table 11 · p. 32, §4.5.2 · p. 32
• On the secondary cohort, model AUCs span 0.270–0.630 for standalone ST and 0.313–0.621 for montage ST_CWT; no model exceeds 0.63, and every confidence interval contains 0.5.Seven of 14 models score higher with montage input and seven score lower.Source: §4.5.2 · p. 32
• On the primary cohort, all fourteen models on standalone CWT images sit at chance (0.307–0.541); on montage CWT_ST images, six reach 0.82 or above, while the proposed CNN reaches 0.874 and ranks fifth.Source: §4.5.2 · p. 31, Table 12 · p. 33, Table 13 · p. 33
• CWT and DWT each reached 0.495 AUC, SSWE 0.479, and ST 0.470 on the secondary cohort; every 95% confidence interval contained 0.5.The four standalone methods sat at chance under stratified group five-fold cross-validation with five random seeds.Source: §4.5.2 · pp. 34–35, Table 15 · p. 35
• 0.457–0.529 was the range for the six secondary-cohort fusion pairings, and every confidence interval contained 0.5.Montage ST_CWT reached 0.514±0.068, versus 0.874 for montage CWT_ST on the primary cohort; these are not the same input, as cohort recording times and montage anchor methods differ.Source: §4.5.2 · p. 35, Table 16 · p. 36
• The subject-level results sit far below segment-level results, measuring the effect of removing patient-level leakage; this study reports both protocols on both cohorts.In segment-level evaluation, segments from one patient can occur in both training and testing; subject-disjoint evaluation keeps each patient in one fold.Source: §4.5.2 · p. 35

4.5.3. Summary of Significance Testing

• p = 0.703 for standalone ST: its observed AUC of 0.500 fell within a null distribution with mean 0.520 and standard deviation 0.083.Seventy of 100 patient-level label permutations reached or exceeded the observed value; the paper says the same chance-range result applies to the other standalone and fused configurations.Source: §4.5.3 · p. 36
• p = 0.005 for primary-cohort montage CWT_ST, whose observed value exceeded all 200 permutations and remained below the corrected 0.00625 threshold.The correction accounts for testing eight fusion configurations.Source: §4.5.3 · p. 36

5.1. Ablation Study - Proposed CNN and Pre-trained CNNs

• The custom CNN uses three convolutional layers with 8, 16, and 32 filters, while the same training setup was applied to baseline models for comparison.The CNN uses 3×3 kernels, ReLU, batch normalization, 2×2 average pooling, dropout 0.25, Adam at 0.0001, binary cross-entropy, and early stopping; pretrained vision models use frozen ImageNet-weighted bases.Source: §5.1 · p. 39

5.2. Proposed CNN Architecture Exploration

• Across eight candidate architectures on standalone CWT images, patient-level AUC was exactly 1.000 under segment-level validation and 0.431–0.493 under subject-disjoint validation, regardless of parameter count (389,121–205,545,057).The passages attribute the inflation to fold construction, not architecture.Source: Table 18 · p. 40, §5.2 · p. 40

5.3. Performance Analysis - Individual TF Methods

• Segment-level validation gave CWT 98.67% accuracy versus DWT 90.22%; under subject-disjoint validation, four methods ranged 0.441–0.485 AUC on the primary dataset and 0.470–0.495 on the secondary dataset.Source: §5.3 · p. 41

5.4. Performance Analysis - Fused TF Methods

• Under segment-level validation, Montage CWT_ST reaches 99.90% accuracy and 99.91% AUC; under subject-disjoint validation, it reaches AUC 0.874 on the primary cohort.The discussion reports that fused representations outperform standalone methods in every segment-level experiment; Montage CWT_ST is the best subject-disjoint configuration.Source: §5 · p. 38, §5.4 · p. 42
• AUC 0.874 ± 0.183 and 82.7% accuracy make Montage CWT_ST on the primary cohort the study’s best result.The discussion identifies the input-level montage configuration as the one that separates responders from non-responders.Source: §5.4 · pp. 42–43

5.5. Methodological Basis for Time-Frequency Fusion

• Montage joins two intact TF images at the input, and the study reports that this approach separates responders when other fusion mechanisms do not.CWT captures fast EEG changes while ST covers frequency and phase more evenly; montage preserves both representations, unlike blending, which overlays them.Source: §5.5 · pp. 43–44

5.6. Comparison with Recent Studies on rTMS Therapy

• 99.90% segment-level accuracy exceeded cited studies, including Shahabi et al. at 99.32%, but the figures are not directly comparable because their fold construction is unspecified.The study reports both segment-level and patient-disjoint evaluations so performance can be interpreted in light of the validation protocol.Source: §5.6 · pp. 44–45

5.7. Limitations

• The study included 15 primary- and 46 secondary-cohort patients; primary-cohort accuracy should be interpreted against its 60.0% majority-class baseline, and small cohorts leave confidence intervals wide.The authors identify confirming montage CWT_ST on a further independent cohort as a natural next step.Source: §5.7 · pp. 45–46

6. Conclusion

• 99.90% accuracy on the primary dataset and 91.90% on the secondary dataset under segment-level validation contrast with performance falling to chance under strict subject-disjoint validation.In segment-level 10-fold cross-validation, segments from the same patient may appear in both training and testing; strict validation keeps each patient's segments in one fold.Source: §6 · p. 47
• Across two cohorts, four representations, six fusion mechanisms, and fourteen architectures, every non-exception configuration had AUC 0.31–0.54 with 95% confidence intervals containing 0.5; the best standalone method's permutation test gave p = 0.703.Source: §6 · p. 47
• AUC 0.874 ± 0.183 and 82.7% accuracy make Montage CWT_ST on the primary cohort the best subject-level result.The authors identify this as the study's best result.Source: §6 · p. 47
• The mechanism behind the result is not yet established, and confirmation of Montage CWT_ST on a larger independent cohort remains a follow-up target.Source: §6 · pp. 47–48
• On cohorts of this size, fold construction changes reported results more than representation, architecture, or fusion, so studies without stated fold construction cannot be interpreted.Source: §6 · p. 47
• Across fourteen architectures, input choice shifted results more than model choice, and Montage CWT_ST was the representation that separated responders from non-responders.The authors point to input construction, rather than network design, as the productive direction for this problem.Source: §6 · pp. 47–48