Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

1 INTRODUCTION

• Structural MRI is a 3D volume whose anatomical and disease-related variations may extend across multiple slices and spatial scales.Effective analysis requires preserving informative volumetric structure while extracting features for downstream prediction.Source: §1 · p. 1
• Dedicated 3D foundation models learn volumetric representations directly, while general-purpose 2D foundation models offer an alternative that can transfer to volumetric medical imaging.Dedicated 3D pretraining requires 3D architectures and substantial medical imaging data.Source: §1 · p. 1
• Existing 2D-to-3D approaches commonly encode slices independently and integrate features afterward, leaving cross-slice information to be incorporated only after slice-level extraction.With a frozen encoder, slice-level representations cannot adapt to facilitate later volumetric aggregation.Source: §1 · p. 2
• MVR transforms each anatomical view from D slices into M ≪ D dense components before encoding, combining an uncentered-PCA base with complementary multiscale residual details.The reduction is estimated from training volumes without diagnostic labels or gradient-based optimization.Source: §1 · pp. 2–3
• MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to evaluated 2D-to-3D adaptation methods and simple input-reduction baselines.Ablations also show consistent contributions from its multiscale construction and base-and-detail components.Source: §1 · p. 3
• MVR’s reduce-then-encode strategy incorporates information across slices before 2D feature extraction rather than only afterward.The resulting component features are concatenated across components and anatomical views.Source: Figure 1 · p. 2, §1 · p. 3

Medical foundation models

• Medical foundation models obtain volumetric representations through pretraining directly on 3D medical images, whereas this paper studies frozen general-purpose 2D models.Examples include BrainIAC, BrainMVP, and 3DINO.Source: §2 · p. 3

Adapting 2D models to medical volumes

• RAPTOR and AnyMC3D integrate volumetric information after slice-wise feature extraction, while MVR integrates it in the input space before encoding.Eigenslices (J¨onemo & Eklund, 2023) also reduces brain MRI to 2D projections but uses a trainable 2D CNN.Source: §2 · p. 3

3.1 OVERVIEW

• MVR represents each axial, coronal, or sagittal view as an H × W × D volume, then constructs M ≪ D 2D components before feature extraction.The base summarizes through-plane intensities, while M − 1 detail components capture complementary multiscale spatial variation.Source: §3.1 · p. 4

3.2 BASE COMPONENT EXTRACTION

• Uncentered PCA projects each location’s D slice intensities onto one shared direction, retaining the dominant through-plane intensity pattern and its in-plane location.Because PCA is uncentered, the projection retains average intensity structure across slice positions.Source: §3.2 · p. 4

3.3 MULTISCALE RESIDUAL EXPANSION

• Gaussian smoothing at ordered scales yields successive-level differences that capture in-plane variation at different spatial scales while preserving through-plane coordinates.The differences are normalized by training-set RMS factors estimated separately for each anatomical view; Figure 3 visualizes the multiscale components.Source: §3.3 · p. 5, Figure 3 · p. 5
• MVR regresses centered multiscale descriptors on centered base scores, then uses PCA to retain the leading orthogonal residual directions as detail components.These details capture residual multiscale variation not linearly predicted by the base score.Source: §3.3 · pp. 4–6

3.4 ENCODING AND CLASSIFICATION

• Fitted reduction statistics remain fixed for unseen volumes; a shared frozen DINOv3 encodes each component independently before feature concatenation and class-balanced, L2-regularized logistic regression.Features are concatenated within each view and then across axial, coronal, and sagittal views.Source: §3.4 · p. 6

4.1 EXPERIMENTAL SETUP

• Evaluation covers Alzheimer’s disease classification on ADNI and OASIS-3, autism classification on ABIDE, and external testing on AIBL, using five-fold evaluation and frozen DINOv3 comparisons.The experiments compare medical 3D models, 2D-to-3D adaptations, and input-reduction baselines; M is fixed at 16 for the five-fold experiments.Source: §4.1 · pp. 6–7

4.2 RESULTS

• MVR achieves the highest AUC on ADNI and OASIS and highest BAcc on ADNI and ABIDE, exceeding evaluated 2D-to-3D adaptation methods across all three datasets.The reported leading values are AUC 88.85 on ADNI and 70.30 on OASIS, and BAcc 80.92 on ADNI and 61.03 on ABIDE.Source: §4.2 · p. 7, Table 1 · p. 7
• On AIBL, MVR achieves the highest AUC and BAcc, 87.27 and 72.47, among the five ADNI-trained models evaluated without AIBL-specific fitting or model selection.Its ADNI advantage is largely maintained on the external cohort.Source: §4.2 · p. 8, Table 2 · p. 8
• With LoRA encoder adaptation, MVR reaches 89.10/81.15 AUC/BAcc, while jointly adapted AnyMC3D reaches 92.93/84.56, bounding MVR’s advantage to the frozen-encoder regime.Primary experiments keep the 2D foundation model frozen.Source: §4.2 · p. 8

4.3 ABLATION STUDY

• Removing Gaussian scale space reduces AUC by 1.84, 0.91, and 2.64 points on ADNI, OASIS, and ABIDE, respectively, supporting multiscale in-plane variation.BAcc drops are 2.58, 3.38, and 1.86 points on those datasets.Source: §4.3 · p. 8
• Retaining only the base component lowers AUC by 3.41, 3.01, and 4.49 points on ADNI, OASIS, and ABIDE; detail components provide larger, more consistent gains.The encoded base component also provides a complementary contribution overall.Source: §4.3 · p. 9, Table 3 · p. 9
• Full concatenation beats mean fusion on ADNI and ABIDE, but mean fusion is slightly better on OASIS at 70.51 AUC and 65.71 BAcc.Concatenation improves AUC by 1.66 and 3.39 points and BAcc by 0.77 and 2.66 points on ADNI and ABIDE, respectively.Source: §4.3 · p. 9
• M = 16 gives the highest AUC and BAcc on ADNI and ABIDE, while OASIS is less sensitive and peaks at M = 3 for AUC/BAcc.Increasing M from 16 to 32 provides no consistent improvement across datasets and metrics.Source: §4.3 · p. 9, Table 4 · p. 9

• MVR reduces volumetric information before encoding and builds complementary base and multiscale detail components for a compact set of dense 2D inputs.The components preserve through-plane structure while incorporating spatial context for a frozen 2D foundation model.Source: §5 · p. 9
• MVR departs from the conventional encode-then-integrate paradigm by reducing volumetric information before foundation-model encoding.Source: §5 · p. 9
• Across multiple brain-MRI cohorts and diagnostic tasks, MVR achieves strong, consistent performance and transfers well to an external cohort.The conclusion presents reduction before encoding as a viable alternative to post-encoding volumetric integration for frozen 2D foundation models.Source: §5 · p. 10

A.1.1 BASE PROJECTION

• The base projection selects a unit direction that maximizes the average squared projection of sampled through-plane vectors.The vectors are stacked as rows of Xv, forming the training matrix used to fit the projection.Source: §A.1.1 · pp. 11–12
• The base direction is the eigenvector of the uncentered second-moment matrix with the largest eigenvalue.The matrix is symmetric and positive semidefinite, so its eigendecomposition supplies the maximizing direction.Source: §A.1.1 · p. 12
• Uncentered PCA retains average through-plane intensity structure, unlike centered PCA, matching the base component’s role as an original-intensity reference.Centered PCA removes the average structure before estimating principal directions.Source: §A.1.1 · p. 12
• Applying the fitted base direction to sampled vectors gives base scores, and the same projection is applied at each in-plane location.The resulting projected values preserve their corresponding in-plane locations.Source: §A.1.1 · p. 12

A.1.2 DETAIL PROJECTION

• Detail fitting uses residual descriptors associated with sampled multiscale descriptors and their covariance matrix.The residual descriptors are stacked as rows in Rv.Source: §A.1.2 · p. 12
• The first detail direction is chosen to retain a direction with large residual variance.Subsequent detail directions are obtained subject to orthogonality constraints.Source: §A.1.2 · p. 12
• The M −1 detail directions are the leading eigenvectors of the residual covariance matrix and are mutually orthogonal in residual PCA space.Applying each direction to a residual descriptor yields its corresponding detail-component value.Source: §A.1.2 · p. 13

Sampling sensitivity

• AUC ranged from 88.33 to 88.85 and BAcc from 79.99 to 81.68 across tested sampling counts; P = 256 led AUC, while P = 512 led BAcc.Using all available vectors did not improve over sampled settings.Source: Table 5 · p. 13, §A.2 · p. 13

Anatomical-view composition

• Three views achieved the highest AUC/BAcc on ADNI (88.85/80.92) and OASIS (70.30/64.49); on ABIDE, three views led AUC (66.02) while axial+coronal led BAcc (61.47).The three-view representation remained competitive on both ABIDE metrics.Source: Table 6 · p. 13, §A.2 · p. 13

LoRA adaptation

• With LoRA rank 8, MVR reached 89.10 AUC and 81.15 BAcc, while AnyMC3D led at 92.93 AUC and 84.56 BAcc.MVR’s primary design instead uses a frozen DINOv3 encoder without gradient-based encoder adaptation.Source: Table 7 · p. 14, §A.2 · p. 14

Gaussian-scale sensitivity

• Increasing Gaussian scales from five to nine changed mean AUC from 88.85 to 88.93, while mean BAcc stayed at 80.92.Both configurations span the same 1–16 mm range.Source: §A.2 · p. 13, Table 8 · p. 14

Feature-layer ablation

• Concatenating Blocks 6 and 12 reached 88.85 AUC and 80.92 BAcc, gains of 0.54 and 0.30 points over Block 12 alone.Block 12 was stronger than Block 6 alone by 1.48 AUC and 2.01 BAcc points; concatenation doubles classifier input dimension without additional DINOv3 forward passes.Source: §A.2 · p. 14

Component scaling and encoder input preparation

• Component scaling statistics are estimated from the training fold, with the 1st and 99th training-value percentiles used for component scaling.Foreground is identified from the normalized volume’s constant minimum-valued background, without an external brain mask or segmentation.Source: §A.3 · p. 14
• Foreground component values are mapped to [0, 1], background is set to zero, and components are replicated across three channels and resized to 256 × 256.The inputs are normalized with DINOv3’s ImageNet channel statistics; scaling and clipping parameters are fixed before validation and test application.Source: §A.3 · pp. 14–15

Feature-layer ablation

• Table 9 reports ADNI feature-layer ablation results using AUC and BAcc on a 0–100 scale across five held-out folds.The caption identifies highlighted entries as the best results.Source: Table 9 · p. 15

Uniform Slice Sampling

• Uniform sampling selects M = 16 slice positions by dividing a fold-level foreground interval into equal-width bins and choosing each bin’s center-nearest slice.The interval boundaries are medians of training-subject occupied-slice boundaries.Source: §A.3 · p. 15
• The selected slice indices are fixed across training and held-out subjects, and their encoded features are concatenated across slices and views.Each slice uses the same training-fold intensity scaling and frozen-encoder preprocessing as MVR components.Source: §A.3 · p. 15