Source-linked AI summary
A Shared-Backbone Approach for Multi-Task MedMNIST Classification
Stefan-Dorian Gavril, Andrei Arhire, Adrian Iftene
TL;DR
MedMNIST’s 11 tasks vary in modality, size, and class balance, while its API offers higher-resolution images than the fixed 28×28 evaluation set. The study fine-tunes pretrained shared convolutional backbones with 11 task-specific linear heads and uses matching 28px-to-64px bicubic upscaling in training and testing. Correcting preprocessing had a larger impact than backbone capacity; afterward ConvNeXt-Tiny (0.73294) outperformed EfficientNet-B0 (0.72287).
Problem
MedMNIST’s 11 tasks vary in modality, size, and class balance, while its API offers higher-resolution images than the fixed 28×28 evaluation set.
Source: §1 · p. 1Method
The study fine-tunes pretrained shared convolutional backbones with 11 task-specific linear heads and uses matching 28px-to-64px bicubic upscaling in training and testing.
Source: §4.1 · p. 3, §4.2 · p. 3Results
Correcting preprocessing had a larger impact than backbone capacity; afterward ConvNeXt-Tiny (0.73294) outperformed EfficientNet-B0 (0.72287).
Source: §7 · p. 6Takeaways & Limitations
Using the same 28px-to-64px bicubic upscaling for training and testing addressed the resolution mismatch that had depressed leaderboard performance.
Source: §8 · p. 6RetinaMNIST remains difficult because its five severity grades depend on fine retinal structures that may be difficult to preserve at 28px, and improvement may require significantly more data or higher native resolution.
Source: §6.4 · p. 6
Abstract
from arXiv · showhide
Multi-task biomedical classification requires models to generalize across disparate modalities and class distributions. We study 11 heterogeneous MedMNIST datasets using the harmonic mean of per-task macro-F1. We evaluate three backbones with task-specific linear heads. We identify a resolution domain shift between the MedMNIST API and evaluation environment. Resolving this inconsistency and optimizing architecture-specific regularization substantially improved performance. Our best configuration, a ConvNeXt-Tiny backbone with label smoothing, achieved a leaderboard harmonic-mean macro-F1 of 0.73294 in the Tensor Reloaded: Multi-Task MedMNIST competition, ranking sixth at the close of the official competition phase. Our implementation is publicly available at: https://github.com/GavrilStefan-Dorian/A-Shared-Backbone-Approach-for-Multi-Task-MedMNIST-Classification
1. Introduction
The paper studies a shared multi-task model on heterogeneous, imbalanced MedMNIST datasets. It highlights small-task performance and inconsistent training and test resolutions as challenges for the benchmark.
1. Introduction
- Multi-task learning could share low-level visual features across modalities while retaining task-specific classification heads, but biomedical datasets vary widely in appearance, size, and class balance.The paper studies this setting using the MedMNIST benchmark and Tensor Reloaded Multi-Task MedMNIST.Source: §1 · p. 1
- Training-image counts run from BreastMNIST to 165,466 for TissueMNIST; the harmonic-mean metric scores models poorly if they ignore small tasks, regardless of large-task performance.The evaluation test set is fixed at 28×28, while the public MedMNIST API provides higher-resolution downloads at 64px, 128px, and 224px.Source: §1 · p. 1
- Training on native 64px API images and evaluating on 28px test images created a resolution domain shift that validation scores could miss.The authors initially trained on native 64px API images and evaluated on the fixed 28×28 test set.Source: §1 · p. 1
- The paper traces the decisions and mistakes behind its final result to identify what moves scores on a heterogeneous, harmonic-mean-scored multi-task benchmark.Source: §1 · p. 1
2. Related Work
Prior work establishes MedMNIST as a lightweight benchmark and studies transfer learning, multi-task sharing, and resolution effects. This paper examines a shared CNN for balanced performance across its tasks.
2. Related Work
- MedMNIST began as a standardized lightweight benchmark, later expanded to 12 two-dimensional datasets, and MedMNIST+ introduced 64-, 128-, and 224-pixel variants [13].The original benchmark reported task variation, including ResNet-18 accuracy of 0.921 on OrganAMNIST and 0.515 on RetinaMNIST at 28px.Source: §2 · p. 1
- ImageNet pretraining offered limited performance gains but improved convergence in evaluated medical-imaging tasks, while shared representations can regularize related tasks.The cited multi-task account notes that gradients from larger tasks can help smaller tasks avoid overfitting despite a 300× TissueMNIST–BreastMNIST size difference.Source: §2 · p. 1
- Task-weighted sampling addresses the risk that large tasks dominate updates in hard-parameter-sharing models [10]; gradient projection and joint loss-and-gradient balancing are other proposed interventions.Source: §2 · p. 2
- CNN fine-tuning at 64px or higher matched or exceeded most alternatives in a MedMNIST+ comparison, while gains beyond some resolutions were inconsistent.The paper relates this to the possibility that resolution interacts with architecture and the train/test pipeline.Source: §2 · p. 2
- This work investigates an efficient shared CNN optimized for balanced performance under harmonic-mean macro-F1, alongside studies of individual MedMNIST tasks and unified Transformer-based multi-task learning [14].Source: §2 · p. 2
3. Dataset Overview
The study covers 11 MedMNIST tasks with varied modalities, dataset sizes, class counts, and class balance. This variation motivates methods that account for task imbalance.
3. Dataset Overview
- Table 1 reports dataset characteristics using Cl. for classes, Imb. for imbalance ratio, Ent. for normalized Shannon entropy, and BF1 for baseline F1.Source: Table 1 · p. 3
- The 11 tasks differ substantially in modality, training-set size, number of classes, and class balance.Representative samples span histopathology, blood-cell microscopy, retinal fundus photography, and CT imaging, despite a shared 28×28 resolution.Source: §3 · pp. 2–3
- DermaMNIST’s majority class has almost 60 times as many samples as its minority class, whereas PathMNIST is nearly balanced.Figure 1 summarizes class imbalance across the datasets.Source: §3 · p. 3
4. Method
The method shares a pretrained convolutional backbone across 11 task-specific heads and evaluates three architectures. It uses consistent image preprocessing, averages task losses, and tests task-balanced sampling and label smoothing.
4.1. Architecture
- A single pretrained convolutional backbone feeds 11 independent linear heads, and the study fine-tunes all parameters for ResNet-18, EfficientNet-B0, and ConvNeXt-Tiny.ImageNet weights come from the timm library.Source: §4.1 · p. 3
4.2. Data Regime and Interpolation
- All Kaggle images are bicubically upscaled to 64×64 in both training and testing to keep image statistics consistent.Native 64px MedMNIST images introduced a domain shift and reduced leaderboard performance; the training pipeline also uses horizontal flipping, ±10° rotation, and per-channel normalization to mean 0.5 and standard deviation 0.5.Source: §4.2 · pp. 3–4
4.3. Multi-Task Training
- Each batch passes through the shared backbone once, and the task-specific cross-entropy losses are averaged.The configuration uses AdamW with weight decay 0.01 and a OneCycleLR schedule peaking at 10^-3; mixed precision is used throughout.Source: §4.3 · p. 4
4.4. Task-Balanced Sampling and Label Smoothing
- Inverse-task-size sample weights balance task contributions in expectation, while label smoothing with ϵ = 0.1 is tested alone and combined with sampling.The two techniques are treated as independent ablation variables because their interaction differs by backbone; oversampling small datasets risks memorization.Source: §4.4 · p. 4
5. Ablation Study and Results
The ablation compares weighted sampling (WS), label smoothing (LS), and pretraining across three backbones, including from-scratch WS+LS references.
5. Ablation Study and Results
- Table 2 reports public-leaderboard harmonic-mean macro-F1 scores for the WS, LS, and pretraining ablations.The comparisons cover three pretrained backbones and include a from-scratch WS+LS reference for each architecture.Source: §5 · p. 4
- The study compares WS and LS across three pretrained backbones.Source: §5 · p. 4
- A from-scratch WS+LS reference is included for each architecture.Source: §5 · p. 4
6. Analysis
Pretraining improves scores across the tested backbones, while the best regularization depends on architecture. Matching training and test resolution recovers ConvNeXt-Tiny performance, but task difficulty remains uneven.
6.1. Pretraining establishes the baseline
- 0.70575 versus 0.64196: pretrained EfficientNet-B0 with weighted sampling and label smoothing outscored its from-scratch counterpart and converged faster.Pretrained weights substantially outperformed training from scratch across all three architectures.Source: §6.1 · p. 4
6.2. Architecture vs. Regularization
- 0.73294 was ConvNeXt-Tiny’s peak with label smoothing alone; ResNet-18 peaked at 0.70471 with label smoothing, while EfficientNet-B0 peaked at 0.72287 with task-balanced sampling.Combining both add-ons was never the single best configuration.Source: §6.2 · pp. 4–5, Table 2 · p. 5
6.3. Fixing the domain shift
- 0.5680 with mismatched resolution rose to 0.7329 when training and test data used the same 28px→64px pipeline.PathMNIST, OCTMNIST, and BreastMNIST saw the largest gains; RetinaMNIST and OrganSMNIST improved least.Source: Table 4 · p. 5, §6.3 · p. 5
6.4. Per-Task Results
- Validation F1 divided tasks into easy (>0.90: BloodMNIST, PathMNIST, OrganAMNIST, OrganCMNIST, PneumoniaMNIST), medium (0.70–0.90: BreastMNIST, OrganSMNIST, OCTMNIST), and hard (<0.70: DermaMNIST, TissueMNIST, RetinaMNIST) groups.RetinaMNIST was worst across every run (0.35–0.44 F1); it involves ordinal regression across five severity grades, and its fine retinal diagnostic features may be difficult to preserve at 28px.Source: §6.4 · p. 6
7. Discussion and Future Work
The experiments identify preprocessing consistency as a larger performance factor than backbone capacity in the mismatched setting. Future work could test native-resolution pipelines, larger backbones, and task-specific or test-time strategies.
7. Discussion and Future Work
- Preprocessing consistency mattered more than backbone capacity: mismatched preprocessing left ConvNeXt-Tiny at 0.5680, while the corrected pipeline enabled a 0.73294 score.With the corrected pipeline, ConvNeXt-Tiny outperformed EfficientNet-B0 (0.72287) and ResNet-18 (0.70471).Source: §7 · p. 6
- A native 64px train-and-test pipeline could avoid upscaling, but would need to handle different test-set sizes.Source: §7 · p. 6
- Larger backbones could test whether capacity gains continue beyond ConvNeXt-Tiny’s 28M parameters.Source: §7 · p. 6
- Per-task loss weighting and test-time augmentation are proposed as further ways to target weak tasks or improve predictions.The proposed loss weighting dynamically upweights failing tasks; test-time augmentation averages predictions across augmented views.Source: §7 · p. 6
8. Conclusion
The paper presents a shared-backbone MedMNIST classifier and reports that correcting the train/test resolution mismatch was a major practical improvement. Its best submission achieved a 0.73294 harmonic mean F1, with RetinaMNIST remaining the main bottleneck.
8. Conclusion
- The paper presents a shared-backbone multi-task classifier and recounts the mistakes and fixes that led to its final result.Source: §8 · p. 6
- 0.73294 harmonic mean F1 was the best submission, while RetinaMNIST remained the main bottleneck because of its small dataset and ordinal label structure.Source: §8 · p. 6
- Using the same 28px-to-64px bicubic upscaling for training and test fixed the domain shift caused by mixing native 64px API images with 28px evaluation images.The mismatch had produced acceptable validation scores but collapsed leaderboard performance.Source: §8 · p. 6