Source-linked AI summary
Sharpening the Ensemble: An SSIM-Aligned Residual Refiner for Brain-MRI Inpainting Post-Processing
Kubilay Kağan Kömürcü, İlkay Öksüz
TL;DR
Brain-MRI inpainting can supply plausible healthy tissue where lesions disrupt the assumptions of automated healthy-brain analysis. This paper applies a structural-similarity-weighted residual refiner to a frozen ensemble and reports a small, consistent SSIM gain at a moderate weight, while noting that single-seed variation may affect comparisons between nearby settings.
Problem
Lesions violate the healthy-anatomy assumptions of automated analysis pipelines, and a healthy baseline for the same patient is rarely available.
Source: §1 · p. 2Method
A residual refiner is trained on the outputs of a frozen two-model ensemble using an ℓ1 loss augmented with a weighted structural-similarity term.
Source: §3 · p. 4, §3.3 · p. 5, §6 · p. 10Results
At λ=0.5, the mean per-case difference was +0.00127 SSIM; 62.6% of 219 cases improved (p=2.2×10^-7), and the leaderboard ordering agreed.
Source: §4.3 · p. 8Takeaways & Limitations
At a moderate structural-term weight, post-processing improves ensemble SSIM on both the held-out scorer reproduction and official validation leaderboard, with little change in pixel-wise metrics.
Source: §6 · p. 10Each configuration uses one training seed, and neighboring λ settings differ by about 10^-3 SSIM, which may not exceed variation from retraining.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Brain-MRI inpainting replaces a masked region of a scan with synthesized, anatomically plausible healthy tissue, so that analysis tools built for healthy brains can be applied to images they would otherwise reject. On the BraTS local-synthesis benchmark, which ranks submissions on the structural similarity index (SSIM), the peak signal-to-noise ratio, and the mean squared error (MSE) jointly, the strongest recent models are accurate, but several report blurry synthesized regions and attribute this to the mean-seeking behavior of the $\ell_1$ and MSE terms in their training losses. We address this in post-processing, forming a deep ensemble of the two co-first-place 2025 models and training a lightweight residual refiner on the ensemble's own outputs under an $\ell_1$ loss augmented with a structural-similarity term whose weight $\lambda$ we vary. At a moderate $\lambda$ the refiner improves SSIM over the ensemble, from $0.8767$ to $0.8780$ on a held-out reproduction of the official scorer and from $0.8555$ to $0.8572$ on the official validation leaderboard, with essentially no change in MSE. The gain is small but consistent, improving $62.6\%$ of the held-out cases with a signed-rank $p=2.2\times10^{-7}$, whereas over-weighting the structural term reverses it. Two ablations bound the effect. Adding any third model to the two-model ensemble degrades it, and classical unsharp masking fails to improve SSIM at any strength (best $0.8765$ against $0.8767$), so the gain reflects learned rather than indiscriminate sharpening. The result is a cheap, reproducible post-processing stage that improves an already strong ensemble without any large-scale retraining.
1 Introduction
Tumor lesions violate the healthy-anatomy assumptions of automated pipelines, while patient-specific healthy baselines are rarely available; inpainting can synthesize missing tissue. Reported blur in leading systems motivates post-synthesis sharpening because SSIM responds directly to lost fine structure within the benchmark’s three-metric ranking.
1 Introduction
- Tumor lesions undermine automated pipelines designed for healthy anatomy, while a healthy baseline for the same patient is rarely available.Inpainting fills a masked pathological region with plausible healthy tissue while preserving surrounding anatomy, enabling healthy-brain pipelines to be applied.Source: §1 · p. 2
- Recent reports link blur to mean-seeking MAE [25] and MSE objectives; BraTS ranks submissions by aggregating SSIM, PSNR, and MSE.SSIM responds most directly to blur because it measures structural agreement and rewards fine structure.Source: §1 · p. 2
- The focused residual refiner produces a small, consistent improvement over the ensemble with an appropriately weighted structural term [14].The study examines post-processing rather than replacing the existing solution, with controlled ablations of structural weight, unsharp masking, and ensemble composition.Source: §1 · p. 2
2 Related Work
Prior work spans several inpainting model families and has explored post-processing pretrained-winner ensembles. This study instead focuses on a structural-similarity-weighted residual refiner applied after a frozen ensemble.
2 Related Work
- Inpainting research includes U-Net regressors, conditional GANs, and diffusion models, and recent BraTS local-synthesis solutions draw on all three families.The cited examples are U-Net, pix2pix, and diffusion models.Source: §2 · p. 3
- Recent BraTS entries span U-Net, conditional diffusion, and GAN approaches, while reports describe residual blur linked to MAE or MSE objectives [25].Zhang et al. [25] attribute residual blur to MAE’s mean-seeking behavior; Ferreira et al. introduce MAE to reduce blur from MSE.Source: §2 · p. 3
- The base models already combine pixel-wise and structural losses; this study examines applying that combination after a frozen ensemble and varying the structural-term weight.Neither base-model report examines that separate post-ensemble network and weight variation.Source: §2 · p. 3
- Unlike Kulkarni et al. [14]’s synthetic-blur MSE enhancement, this refiner uses actual ensemble outputs, predicts a residual, inputs model disagreement, and up-weights SSIM.Kulkarni et al.’s method was officially ranked while post-processing pretrained winners, establishing a precedent for the eligibility of this approach class.Source: §2 · p. 3
- Two additional 2025 entries, Local2Global and PSegGAN, are evaluated later as third members of the ensemble.Local2Global is a U-Net with hierarchical attention; PSegGAN is conditioned on a pseudosegmentation of surrounding tissue.Source: §2 · p. 3
- The pipeline averages predictions from two frozen 2025 co-winners, then adds a learned residual correction conditioned on the inputs, including model disagreement.The base models are Zhang et al. [25] and Ferreira et al. [8]; neither is fine-tuned.Source: Figure 1 · p. 4
3 Methods
The pipeline freezes the two co-first-place 2025 predictors as a mean ensemble, then trains a residual refiner with a varied structural-loss weight; unsharp masking provides a nonlearned comparison.
3.1 Base Models and Deep Ensemble
- The two frozen 2025 base models are averaged voxel-wise; averaging reduces variance and improves every reported metric over either model alone.The base models are used as released, without fine-tuning.Source: §3.1 · p. 4, §3 · p. 4
3.2 Residual Refiner
- The refiner predicts a residual added to the ensemble mean, using the voided image, both predictions, their disagreement, and the void mask as inputs.The intensities are z-scored using statistics of the voided image, with the same preprocessing during training and inference.Source: §3.2 · pp. 4–5
3.3 Training Objective
- The training loss combines ℓ1 with a λ-weighted structural term, balancing the smoothing tendency of conditional-mean predictions against preserving local contrast and structure.The loss is evaluated on the sub-region used by the official scorer.Source: §3.3 · p. 5
3.4 Unsharp Masking Baseline
- Unsharp masking serves as a nonlearned baseline by adding a scaled high-frequency residual—the ensemble minus its Gaussian-blurred copy—to the ensemble.Setting a = 0 recovers the ensemble.Source: §3.4 · p. 5
4 Results
The two-model ensemble was the strongest tested base configuration, and a λ=0.5 residual refiner produced a small, consistent SSIM gain over it. Higher structural weighting reversed the gain, while unsharp masking did not improve SSIM.
4.1 Setup
- Evaluation used 219 official validation cases and a separate 219-case held-out training split; the official validation ground truth was not released.The scorer evaluates only the healthy portion of the voided region; the base models had been trained on the pool containing the held-out split.Source: §4.1 · pp. 5–6
4.2 Ensemble Composition
- The two-model mean improved on either base model, while adding Local2Global, PSegGAN, or the 2024 winner degraded the ensemble.The reported SSIM costs were −0.0423, −0.0084, and −0.0033, respectively.Source: §4.2 · p. 6, Table 1 · p. 7
4.3 Effect of the Structural Weight
- λ=0.5 had the highest SSIM in both the held-out sweep and official leaderboard, whereas λ=5 fell below the ensemble in both evaluations.The leaderboard cases were not used for training or tuning.Source: §4.3 · p. 7, Table 2 · p. 7, Table 3 · p. 8
- At λ=0.5, mean per-case SSIM difference was +0.00127 (95% bootstrap interval [+0.00071, +0.00182]); 62.6% of 219 cases improved (Wilcoxon p=2.2 × 10^-7).The interval excludes zero; λ=0.1 was not distinguishable from the ensemble.Source: Table 4 · p. 8, §4.3 · p. 8
4.4 Comparison with Unsharp Masking
- Unsharp masking's best SSIM was 0.8765 versus ensemble 0.8767, while the refiner reached 0.8780 without increasing MSE.Unsharp-masking MSE increased monotonically with the amount applied.Source: §4.4 · pp. 8–9, Table 5 · p. 9
5 Discussion
The discussion attributes whether refinement helps to the structural-loss weight, and contrasts learned refinement with unsharp masking. It also cautions that the gains are small and may be sensitive to training seed.
5 Discussion
- The structural-term weight, rather than refiner capacity, governs improvement: ℓ1 dominance preserves smoothing, while excessive structural weight over-sharpens and lowers score.The intermediate settings have the same ordering on the held-out reproduction and official validation leaderboard; Zhang et al. [25] report poor masked-region preservation and boundary artifacts under a purely structural objective.Source: §5 · p. 10
- Only the learned refiner improves SSIM; unsharp masking instead increases MSE, consistent with the refiner adding anatomy-related rather than indiscriminate high-frequency content.Source: §5 · p. 10
- Single-seed training limits confidence in neighboring λ comparisons because their roughly 10^-3 SSIM differences may be smaller than variation from retraining.The leaderboard results use cases unseen by the pipeline and are the evidence least affected by this concern; transfer to another ensemble is untested.Source: §5 · p. 10
6 Conclusion
The paper applies a residual refiner to the ensemble of two co-first-place 2025 models and varies the structural-similarity loss weight. At a moderate weight, it improves SSIM on both evaluations with little change in pixel-wise metrics, while the ℓ1-dominated setting is indistinguishable from the ensemble.
6 Conclusion
- The moderate-weight refiner improves SSIM over the ensemble on both held-out scorer reproduction and official validation, with little change in pixel-wise metrics.Source: §6 · p. 10
- A residual refiner combines ℓ1 and structural-similarity loss to post-process an ensemble of two brain-MRI inpainting models; at moderate similarity-term weight, it improves SSIM over the ensemble on held-out official-scorer reproduction and the official validation leaderboard, with little pixel-metric change.The improvement was small but consistent across cases, without retraining any base model.Source: §6 · p. 10
- The improvement is small but consistent across cases; the setting dominated by ℓ1 is not distinguishable from the ensemble.Source: §6 · p. 10