PaperlayerView exact revision on arXiv

Source-linked AI summary

Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

Chyong Yi Poh, Hwa Hui Tew, Junn Yong Loo, Raphaël C. -W. Phan, Fuad Noman, Pew-Thian Yap, Chee-Ming Ting

arXiv:2610.02722v1cs.LG

TL;DR

Paired SC–FC data are scarce, and existing generators often miss higher-order interactions or generate the modalities independently. MHG-FM uses modality-specific hypergraphs, bidirectional fusion, and latent flow matching for joint generation and translation; it achieves strong fidelity and topology preservation with approximately 8× faster generation than a matched diffusion backbone.

  • Problem

    Paired SC–FC data are challenging and costly to acquire, while existing generators often use pairwise graphs and generate modalities independently, limiting higher-order and coupling modelling.

    Source: §I · p. 1

  • Method

    MHG-FM combines modality-specific hypergraph encoders, bidirectional DCA fusion, and conditional flow matching in a shared variational latent space for joint generation and translation.

    Source: §I · p. 2

  • Results

    MHG-FM achieves strong matrix fidelity and the best overall topology preservation among evaluated methods, with approximately 8× faster generation than the matched MHG-DiT backbone.

    Source: §IV · p. 10

  • Takeaways & Limitations

    The same trained model supports bidirectional SC–FC prediction without retraining or iterative ODE integration.

    Source: §IV · p. 10

Abstract

from arXiv · show

Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely used in neuroimaging studies of neuropsychiatric disorders. Generative modelling can alleviate the scarcity of large-scale paired SC-FC data, but existing approaches typically use pairwise graphs that capture only dyadic interactions and often generate SC and FC independently, limiting preservation of higher-order structure-function relationships. We propose a Multimodal Hypergraph Flow Matching (MHG-FM) framework for joint SC-FC connectivity generation and cross-modal translation. MHG-FM constructs modality-specific hypergraphs, learns higher-order representations with Hypergraph Neural Network (HGNN) encoders, and performs bidirectional cross-modal fusion using Dual Cross-Attention (DCA). A variational autoencoder maps the fused representations to a compact latent space, where conditional flow matching enables connectivity synthesis and multimodal translation via latent transport. Experiments on the Human Connectome Project Young Adult (HCP-YA) dataset show that MHG-FM outperforms several state-of-the-art baselines in reconstruction quality, topology preservation, distributional similarity, and SC-FC coupling, while achieving approximately 8x faster sampling than a matched diffusion backbone.

I. INTRODUCTION

Paired SC–FC data are difficult to acquire at scale, while existing generative approaches often miss higher-order interactions and joint structure–function relationships. MHG-FM addresses these gaps with hypergraph representations, bidirectional cross-modal fusion, and latent flow matching.

I. INTRODUCTION

  • Paired SC–FC neuroimaging data are challenging, costly, and time-consuming to acquire, often leaving limited sample sizes.Joint modelling matters because SC and FC provide complementary perspectives, and their coupling is an important characteristic of brain organisation.Source: §I · p. 1
  • Existing brain-connectivity generators use pairwise graphs and often generate SC and FC independently, limiting higher-order modelling and preservation of SC–FC coupling.Earlier graph generative models also faced training-stability and optimization challenges.Source: §I · p. 1
  • Hypergraph studies have largely been discriminative, while existing hypergraph generation and flow matching have not addressed multimodal coupling together.HYGENE is not designed for multimodal generation or coupling preservation, and diffusion models typically require costly iterative denoising.Source: §I · p. 1
  • MHG-FM jointly generates SC–FC connectomes and supports bidirectional cross-modal translation using modality-specific hypergraphs, HGNN encoders, DCA fusion, and conditional flow matching in a variational latent space.The shared latent representation is used to learn continuous transport trajectories for synthesis and translation.Source: §I · p. 2

II. METHODS

  • The framework models SC and FC as separate hypergraphs, fuses them through bidirectional cross-modal attention, and learns their joint generative distribution with flow matching in a shared latent space.The design targets higher-order dependencies, within-modality fidelity, and structure–function coupling.Source: §II · p. 2

A. Problem Formulation

The model learns a joint distribution over paired SC and FC adjacency matrices, using hypergraphs induced from the observed connectomes as fixed structural scaffolds. SC hyperedges reflect anatomical connections, while FC hyperedges combine statistical motifs.

A. Problem Formulation

  • Each subject contributes paired SC and FC adjacency matrices as training data and reconstruction targets.The matrices represent connectivity over a common set of brain regions and are derived from DTI tractography and resting-state fMRI.Source: A. Problem Formulation · pp. 2–3
  • The objective is to learn a parameterized joint distribution over paired connectomes while preserving structure–function coupling and global topology, and supporting cross-modal translation.The induced hypergraphs are structural priors for message passing, not reconstruction targets.Source: A. Problem Formulation · p. 3

B. Multimodal Hypergraph Construction

  • SC hyperedges are formed by thresholding population-mean structural connectivity, grounding their membership in measured white-matter connections.This contrasts with FC hyperedges, which are built from statistical motifs.Source: B. Multimodal Hypergraph Construction · p. 3
  • FC hyperedges pool triangle cliques, hub-and-spoke groups, and k-nearest-neighbour groups at k = 5 and k = 10, then remove exact duplicates.The triangle threshold is r_tri = 0.7; hub-and-spoke groups use top 80th-percentile-degree hubs and their k_star = 8 strongest correlates.Source: B. Multimodal Hypergraph Construction · p. 3
  • Population-level joint incidence matrices retain hyperedges found in at least 30% of training subjects through majority voting.This retention threshold balances robustness against inter-subject variability.Source: B. Multimodal Hypergraph Construction · p. 3
  • The SC and FC hypergraphs are computed once from their adjacency matrices and remain fixed scaffolds for HGNN message passing, not a second generative stage.They let the model aggregate information across groups of related regions while keeping the generated outputs as N × N adjacency matrices.Source: B. Multimodal Hypergraph Construction · p. 3

C. Multimodal Encoding & Dual Cross-Attention Fusion

Separate SC and FC encoders produce modality representations, which DCA fuses through bidirectional information exchange. The fused embedding is projected into a shared variational latent space.

C. Multimodal Encoding & Dual Cross-Attention Fusion

  • The SC encoder uses two residual HGNN layers with batch normalization before mean pooling, while the FC encoder uses two dual-stream blocks with intra-block cross-attention.The SC encoder's two-layer depth balances representational capacity against over-smoothing; residual connections retain original features and batch normalization stabilizes training.Source: C. Multimodal Encoding & Dual… · p. 3
  • DCA exchanges information bidirectionally between SC and FC representations, with each attention direction using one modality's query and the other modality's keys and values.The design is motivated by structure–function coupling, with residual connections and layer normalization applied before the fused representations are projected by an MLP.Source: C. Multimodal Encoding & Dual… · p. 3
  • The fused embedding is mapped to a 128-dimensional Gaussian latent representation using the reparameterization trick, with KL regularization annealed to 5 × 10^-5 over 9,000 steps.The four-way concatenation preserves attended cross-modal and original unimodal representations before projection.Source: C. Multimodal Encoding & Dual… · p. 3

D. Joint Multimodal Latent Flow Matching

Conditional flow matching learns continuous-time transport from Gaussian noise to the joint multimodal latent target. A linear interpolation yields a tractable training objective, while the continuous-time velocity-field formulation can reduce sampling complexity relative to diffusion.

D. Joint Multimodal Latent Flow Matching

  • Flow matching learns a continuous-time velocity field transporting samples from an isotropic Gaussian source to the joint multimodal latent target.The target latent is formed from the two modality latents.Source: D. Joint Multimodal Latent Flow Matching · p. 3
  • A time-dependent flow map transports source samples toward the target, with intermediate state x_t evolving according to the parameterized velocity field.The flow map induces a time-dependent probability path by pushing the source distribution through the transport.Source: D. Joint Multimodal Latent Flow Matching · p. 4
  • The probability path and its target velocity field satisfy the continuity equation to conserve probability mass during transport.Source: D. Joint Multimodal Latent Flow Matching · p. 4
  • Because direct optimization is generally intractable, the method uses a rectified linear interpolation path and its conditional target velocity to form a tractable objective.The interpolation is x_t = (1 − t)x_0 + tx_1, whose time derivative gives the target velocity.Source: D. Joint Multimodal Latent Flow Matching · p. 4
  • The continuous-time velocity-field formulation can reduce sampling complexity while maintaining generation fidelity compared with diffusion frameworks requiring hundreds to thousands of denoising steps.Source: D. Joint Multimodal Latent Flow Matching · p. 4

E. Modality-Specific Hypergraph Decoding

Separate SC and FC decoders map the joint latent representation to connectivity matrices using modality-specific prediction pathways.

E. Modality-Specific Hypergraph Decoding

  • The SC decoder expands latent z into N × 256 node features through two HGNN layers, then predicts edge weights with a symmetric pairwise MLP and sigmoid×3 activation.The activation constrains outputs to (0, 1).Source: E. Modality-Specific Hypergraph Decoding · p. 4
  • The FC decoder blends local HGNN pairwise predictions with a global MLP prediction using a learned gate.The gate is defined as σ(Wgz) and clamped to [−1, 1].Source: E. Modality-Specific Hypergraph Decoding · p. 4

F. Training Objective

The training objective combines modality-specific reconstruction with flow-matching, KL, and sparsity terms, with fixed loss weights selected on the validation set.

F. Training Objective

  • The objective has five components, including topology-weighted SC MSE with a Pearson-correlation auxiliary term and FC MSE with its own Pearson auxiliary term.The Pearson auxiliary weights are λr = 3.0 for SC and λr = 0.5 for FC.Source: F. Training Objective · p. 4
  • The sparsity term promotes SC sparsity consistent with white-matter tract biology.The loss weights were selected on the validation set and fixed across experiments, with larger weights on modality-specific reconstruction terms than on flow matching, KL, and sparsity regularization.Source: F. Training Objective · p. 4

G. Multimodal Flow Matching Sampling and Generation

At inference, the model integrates a learned velocity field to generate connectomes, while cross-modal translation uses a deterministic encoder–decoder pass.

G. Multimodal Flow Matching Sampling and Generation

  • De novo generation initializes x0 from a standard Gaussian and integrates the learned velocity field over t ∈ [0, 1].The resulting latent is split into SC and FC halves for their modality-specific decoders.Source: G. Multimodal Flow Matching Sampling… · p. 4
  • Cross-modal translation zero-fills the unavailable modality, fuses the observed modality representation through DCA, and decodes only the missing modality without ODE sampling.The incidence matrices remain fixed during translation.Source: G. Multimodal Flow Matching Sampling… · pp. 4–5
  • Fine-tuning uses modality dropout with p∅ = 0.25 so translation inputs match input patterns the model was trained to handle.Each batch contains both modalities with probability 1 − 2p∅, or one zero-filled modality with probability p∅.Source: G. Multimodal Flow Matching Sampling… · p. 5

III. EXPERIMENTAL RESULTS

On HCP-YA, MHG-FM provides more consistent topology preservation than MHG-DiT while maintaining competitive matrix fidelity; experiments use a 562/70/70 subject split.

A. Experimental Setup and Implementation Details

  • The HCP-YA 3T evaluation uses AAL-116 parcellation and 562/70/70 subjects for training, validation, and testing.SC comes from dMRI tractography and FC from resting-state fMRI correlations.Source: A. Experimental Setup and… · p. 5
  • Baselines share the proposed method’s train/validation/test split, and topology evaluation includes distributional and community-structure measures.For unconditional generation, NMI and ARI are averaged over random permutations because generated and real samples lack subject-wise correspondence.Source: A. Experimental Setup and… · p. 5

B. Generation Quality

  • MHG-DiT achieves the highest SC accuracy with MHG-FM comparable; MHG-FM leads multimodal FC accuracy in both settings, while W1 favors FM unimodal and DiT joint.No external baseline exceeds an R2 of 0.28 for FC.Source: B. Generation Quality · p. 5, Table I · p. 6
  • MHG-FM provides the strongest overall topology preservation, with lowest Deg-W1 for SC and FC and best FC Edge-W1 and NMI; MHG-DiT achieves the smallest |∆MI|.MHG-FM also outperforms all external joint baselines on SC–FC coupling except WGAN-Joint.Source: B. Generation Quality · p. 6, Table II · p. 7
  • Both backbones recover modular SC organization and block-diagonal FC structure in averages across the 70 test subjects, with residual errors concentrated around a few high-degree edges.The figure compares ground truth, generated matrices, and residuals for SC and FC.Source: B. Generation Quality · p. 6, Figure 2 · p. 7

C. Ablation Study

The ablations favor the full joint model on FC accuracy and SC–FC coupling while showing trade-offs in SC metrics and benefits associated with cross-attention, AdaLN conditioning, and HGNN encoders.

C. Ablation Study

  • Removing velocity-backbone cross-attention slightly improves SC accuracy but lowers FC accuracy and coupling; additive conditioning instead of AdaLN worsens all FC metrics.MHG-A1 retains DCA fusion, while MHG-A2 retains DCA and flow matching.Source: C. Ablation Study · pp. 7–8
  • MHG-FM is the only joint variant best simultaneously on FC accuracy and coupling, with the lowest |∆MI|, despite lower SC accuracy than MHG-A1 and MHG-Score.Table III ranks the five joint variants; unimodal rows are excluded.Source: C. Ablation Study · p. 8, Table III · p. 8
  • MHG-Score has the best SC MAE and R2 among joint variants, but a larger coupling gap than MHG-FM.It is also closest to MHG-FM on FC, while MHG-A1 attains marginally higher SC r.Source: C. Ablation Study · pp. 7–8
  • Replacing HGNN encoders with pairwise graph convolution makes MHG-GNN underperform MHG-FM across all metrics under the same training setup.Unimodal pathways cannot be evaluated for SC–FC coupling because they generate only one modality.Source: C. Ablation Study · p. 8

D. Training Stability and Statistical Significance

Across three seeds, MHG-FM’s reported variation is smaller than subject-level variability. Paired testing finds complementary SC and FC strengths for MHG-DiT and MHG-FM, with several metrics showing no significant difference after correction.

D. Training Stability and Statistical Significance

  • SC Pearson r across seeds was 0.9584, 0.9562, and 0.9573 (mean±SD 0.9573±0.0009); FC Pearson r was 0.8165, 0.8063, and 0.8187 (mean±SD 0.8138±0.0054).These variations were substantially smaller than subject-level variability reported in Table I.Source: D. Training Stability and Statistical… · p. 8
  • MHG-DiT significantly favors SC MAE, R2, Frobenius norm, and W1, while MHG-FM significantly favors FC r and FC W1.The effect-size figure reports matched-pairs rank-biserial correlation for n = 70; positive values favor MHG-FM, and filled markers denote corrected significance.Source: D. Training Stability and Statistical… · p. 8, Figure 3 · p. 8
  • No significant differences remain after Holm–Bonferroni correction for SC r, FC MAE, FC R2, FC Frobenius norm, or the SC–FC coupling gap.The paired Wilcoxon signed-rank tests used 70 subjects and corrected across eleven comparisons.Source: D. Training Stability and Statistical… · p. 8

E. Computational Efficiency

MHG-FM's matched-diffusion comparison reports faster generation alongside a more favorable balance of fidelity and topology. The same trained model supports bidirectional SC–FC translation without retraining or iterative ODE integration.

E. Computational Efficiency

  • 32.0 vs. 250 evaluations and 106.2 vs. 848.2 ms/sample show MHG-FM's advantage over matched MHG-DiT, although single-pass baselines remain faster in absolute terms.Both iterative methods use the same 14.045M-parameter DCA-fusion backbone and have nearly identical peak memory; MHG-FM uses an adaptive ODE solver with variable evaluations, unlike MHG-DiT's fixed NFE.Source: E. Computational Efficiency · p. 9

F. Cross-Modal Translation

  • In FC→SC translation, MHG-FM achieves the best overall performance and reduces Frobenius error by more than 40% over topoGAN.MHG-DiT consistently ranks second in this direction.Source: F. Cross-Modal Translation · p. 9
  • In SC→FC translation, MHG-FM leads in r, MAE, Frobenius error, and W1, while MHG-DiT attains the highest R2.SC→FC is more challenging for all methods; one trained model enables both translation directions without retraining or iterative ODE integration.Source: F. Cross-Modal Translation · p. 9

IV. CONCLUSION

  • Approximately 8× faster generation than matched MHG-DiT accompanies strong matrix fidelity, best overall topology preservation, and support for both generation and translation.The conclusion also reports ablation evidence for cross-modal attention and hypergraph encoding contributions to multimodal representation and SC–FC coupling.Source: §IV · pp. 9–10
  • Evaluation is limited to the HCP-YA 3T cohort; generalization to independent clinical and multi-site datasets remains future work.Source: §IV · p. 10

Related papers

Loading 2610.02722v1…