Source-linked AI summary
BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models
Junfeng Xia, Wenhao Ye, Junxiang Zhang, Jiayu Zuo, Mo Wang, Quanying Liu
TL;DR
fMRI foundation models commonly mix heterogeneous pretraining domains and adapt downstream tasks independently, leaving unclear how to allocate training experiences and supervision. The paper measures domain and task relations to guide cumulative pretraining curricula and budgeted transfer planning. The joint curriculum improves three reconstruction metrics relative to uniform sampling, while taskonomy-guided policies show descriptive gains when higher-order routes are available.
Problem
fMRI foundation models commonly treat heterogeneous pretraining domains as flat mixtures and adapt downstream tasks independently, despite the need to allocate training experiences and limited supervision.
Method
The paper measures domain difficulty and directed facilitation to build a cumulative curriculum, then uses first- and higher-order task transfer with BIP to select supervised sources and target-specific routes.
Results
6.5%, 16.3%, and 10.5% reductions in v-NMSE, PSD-NMSE, and FC-MSE, respectively, are reported for the joint priority-domain and high-to-low-timestep curriculum versus uniform sampling over both dimensions.
Takeaways & Limitations
Measured learning relations can organize fMRI pretraining and downstream adaptation without changing the backbone, with asymmetric transfer supporting target-specific supervision policies.
Takeaways & Limitations
Policy validation remains exploratory, and task-dependent SALD and PPMI results show that improved in-domain organization does not ensure uniform out-of-domain robustness.
Abstract
from arXiv · showhide
fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.
Introduction
The paper argues that fMRI foundation-model training should use measured relations among heterogeneous domains and downstream tasks rather than treating each as an independent flat set. It introduces relation-guided curricula for pretraining and taskonomy-guided supervision and transfer.
- fMRI foundation models face allocation questions about which training experiences to prioritize and which capabilities to supervise directly for transfer.
- Pretraining domains are often mixed from the beginning despite differences in dynamics, populations, acquisition, signal quality, and scale.
- The study characterizes fixed-budget difficulty and directed facilitation across ten fMRI domains, plus first- and higher-order transfer across fifteen downstream tasks.
- The framework uses directed domain relations to construct a cumulative curriculum with high-to-low-noise timestep scheduling and joint consolidation.
- Budgeted integer programming selects supervised source tasks and target-specific transfer routes from the downstream taskonomy under limited supervision.
- Experiments report improved pretraining fidelity and in-domain transfer, while exploratory sealed-test results suggest benefits from higher-order taskonomy-guided policies.
Related Work
Related work frames heterogeneous neuroimaging data as structured learning material rather than merely nuisance variation. Prior methods address domain sampling, skill prerequisites, timestep curricula, and directed task transfer, while Table 1 evaluates joint timestep and domain policies.
- Foundation models span ROI- and voxel-level fMRI objectives, including masked-signal reconstruction, latent prediction, connectivity reconstruction, representation alignment, and dynamic modeling.
- Prior data-mixture methods optimize domain proportions using proxy or cross-domain signals, while Skill-It uses prerequisite relations and diffusion curricula schedule timesteps by convergence difficulty.
- fMRI heterogeneity includes brain states, populations, diagnoses, acquisition protocols, resolutions, and preprocessing spaces.
- Table 1 crosses timestep policies in panels with uniform, random, and priority-guided domain policies in rows, reporting lower-is-better reconstruction errors.
- Asymmetric transfer across spaces and populations, task interdependence, and source-composition effects motivate modeling transfer beyond pairwise relations.
Method
The method organizes fMRI pretraining and downstream adaptation using measured domain difficulty, directed facilitation, and task transfer, without modifying the Brain-DiT backbone. It combines nested curriculum scheduling with budget-aware selection of supervised sources and target-specific transfer routes.
- Structure-Guided Curriculum Pretraining: Domain difficulty and directed facilitation determine a priority-guided order for introducing fMRI domains.Domains are defined as dataset–state pairs and prioritized using proxy-model difficulty together with their mean outgoing facilitation.
- Structure-Guided Curriculum Pretraining: Directed facilitation compares compute-matched source-first policies with generic non-target pretraining on a subsequent target domain.Positive facilitation indicates better target preparation than the reference policy, whereas negative facilitation indicates a relative disadvantage; it is not a leave-one-domain-out marginal contribution.
- Structure-Guided Curriculum Pretraining: The curriculum progressively expands from high-noise to low-noise timestep clusters while retaining previously activated domains and clusters.Plateau gates trigger new domains and then lower-noise clusters, followed by uniform joint consolidation across all domains and timesteps.
- Downstream Taskonomy: First-order transfer compares each source-task route with a matched target self-transfer reference, while higher-order routes concatenate PCA-projected source representations for a target readout.Higher-order routes use order-matched self-transfer references with matched fusion, head capacity, support, initialization, and optimization budgets.
- Budget-Aware Transfer Planning: BIP selects exactly K directly supervised source tasks and one route per target using construction-validation taskonomy utilities.Route assignments require their source tasks to receive supervision, and maximum-order policies may still select lower-order routes because route spaces are nested.
Experiments
Experiments show that measured domain difficulty and directed facilitation improve pretraining organization, while taskonomy-based transfer reveals asymmetric routes and supports budget-aware source selection. Curriculum gains extend to in-domain evaluations, with selective and exploratory benefits on held-out and higher-order transfer settings.
- Domain learning relations: The ten fMRI domains differ by 2.65× in proxy difficulty, and facilitation is strongly asymmetric across domains.HCP task provides 4.1% and 3.8% directed facilitation for ADNI and NKI, respectively, while ADNI provides little or negative benefit for most domains.
- Pretraining curriculum: The high-to-low Priority curriculum achieves the best v-NMSE, PSD-NMSE, and FC-MSE, improving over uniform domains and timesteps by 6.5%, 16.3%, and 10.5%.It combines timestep scheduling with a priority-guided cumulative domain order; timestep scheduling explains most aggregate gain, while domain order adds complementary benefit.
- Downstream evaluation: The high-to-low Priority configuration is strongest on all four in-domain tasks and outperforms BrainLM, Brain-JEPA, and BrainMass.Reported results include ABIDE age MSE 0.43 with r = 0.71 and HCP sex accuracy 84.32% with macro-F1 84.29%.
- Out-of-domain evaluation: Held-out transfer is selective: Random domain leads SALD MSE, Uniform domain leads SALD correlation, while Priority leads held-out PPMI accuracy and macro-F1.SALD results therefore do not identify one universally dominant domain policy.
- Taskonomy: Downstream transfer is asymmetric and target dependent, with positive transfer on 39% of within-dataset edges versus 32% of cross-dataset edges.Reversing source and target generally changes the result, and neither dataset context nor task semantics fully explains the observed structure.
- Budget-aware transfer: BIP concentrates small-budget supervision on reusable sources, adds specialized sources with larger budgets, and gains descriptively from higher-order routes mainly through route selection.HCP PMAT and CHCP seven-task decoding are the most frequently reused hubs, appearing in 84 and 67 selected routes.
Discussion and Conclusion
Measured learning relations organize both fMRI pretraining and downstream adaptation, but the reported comparisons emphasize endpoint quality and exploratory transfer evidence rather than matched-compute efficiency or uniform OOD robustness.
- Priority-guided domain ordering and high-to-low-noise timestep scheduling jointly improve reconstruction quality over flat domain mixing.The two curricula address complementary dimensions: which experiences appear first and denoising difficulty.
- Random cumulative ordering improves over flat domain mixing, while priority guidance adds gains most clearly in v-NMSE and spectral fidelity.
- Timestep scheduling accounts for most aggregate reconstruction improvement, whereas domain ordering determines which experiences are introduced first.
- Endpoint comparisons do not establish matched-compute efficiency because different policies plateau after different update counts.
- Downstream transfer is asymmetric and target-dependent; BIP yields descriptive gains with higher-order routes, but improved in-domain organization does not ensure uniform OOD robustness.Policy validation remains exploratory, and task-dependent SALD and PPMI results preserve this scope boundary.