PaperlayerView exact revision on arXiv

Source-linked AI summary

LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learning

Xunkai Li, Zekai Chen, Zhengyu Wu, Henan Sun, Daohan Su, Guang Zeng, Hongchao Qin, Rong-Hua Li, Guoren Wang

arXiv:2608.24795v1cs.LG

TL;DR

Existing MAG methods inadequately incorporate graph context during modality alignment and fail to adapt fusion to topology-aware aligned tokens. LION addresses this with Clifford-geometric propagation followed by adaptive holographic aggregation, and reports average improvements of 5.24% and 7.68% over SOTA baselines on graph and modality tasks. Its theoretical guarantees are scoped to bounded constructions and fixed-operator conditions, while experiments use the prevalent two-modality text-image setting.

  • Problem

    Existing MAG methods neglect graph context during modality alignment and inadequately exploit topology priors in aligned tokens during modality fusion.

  • Method

    LION constructs a Clifford-algebra geometric manifold, uses CGP for topology-aware high-order modality interaction, and applies AHA for adaptive fusion.

  • Results

    5.24% and 7.68% average improvements over SOTA baselines are reported for graph and modality tasks, respectively, across evaluations from 6 domains.

  • Takeaways & Limitations

    LION provides a unified alignment-then-fusion paradigm that represents topology and modality within a shared Clifford geometric space.

  • Takeaways & Limitations

    Theoretical contraction is conditional on fixed reciprocal operators, and empirical generality beyond the two-modality text-image setting is not claimed.

Abstract

from arXiv · show

Recently, the rapid advancement of multimodal domains has driven a data-centric paradigm shift in graph ML, transitioning from text-attributed to multimodal-attributed graphs. This advancement significantly enhances data representation and expands the scope of graph downstream tasks, such as modality-oriented tasks, thereby improving the practical utility of graph ML. Despite its promise, limitations exist in the current neural paradigms:(1) Neglect Context in Modality Alignment: Most existing methods adopt topology-constrained or modality-specific operators as tokenizers.These aligners inevitably neglect graph context and inhibit modality interaction, resulting in suboptimal alignment.(2) Lack of Adaptation in Modality Fusion: Most existing methods are simple adaptations for 2-modality graphs and fail to adequately exploit aligned tokens equipped with topology priors during fusion, leading to poor generalizability and performance degradation.To address the above issues, we propose LION (c\underline{LI}ff\underline{O}rd \underline{N}eural paradigm) based on the Clifford algebra and decoupled graph neural paradigm (i.e., propagation-then-aggregation) to implement alignment-then-fusion in multimodal-attributed graphs. Specifically, we first construct a modality-aware geometric manifold grounded in Clifford algebra.This geometric-induced high-order graph propagation efficiently achieves modality interaction, facilitating modality alignment.Then, based on the topology-aware Clifford components of aligned tokens, we propose adaptive holographic aggregation. This module integrates component-wise energy and propagation-scale information with learnable parameters to improve modality fusion. Extensive experiments on 9 text-image MAG datasets demonstrate that LION significantly outperforms SOTA baselines across 3 graph and 3 modality downstream tasks.

I. INTRODUCTION

Multimodal-attributed graphs broaden graph learning but expose alignment and fusion limitations in existing neural paradigms. LION addresses these issues by combining Clifford-geometric propagation with adaptive holographic aggregation, achieving strong gains across graph and modality tasks.

  • Motivation: MAGs represent real-world entities with multimodal data, expanding node-feature semantics and supporting graph-centric and modality-oriented tasks.These tasks include node classification, link prediction, clustering, retrieval, and generation.
  • Limitations: Existing alignment methods neglect broader graph context, with topology-constrained approaches often restricted to 1-hop neighborhoods and unable to capture long-range dependencies.The paper argues that modality-aware, high-quality topology is needed for deeper modality interaction and alignment.
  • Limitations: Existing fusion methods inadequately exploit topology priors in aligned tokens, while many approaches treat topology as auxiliary context rather than integrating it with modality in a shared space.This limits the use of topology for long-range and cross-modality dependencies.
  • LION: LION uses Adaptive Holographic Aggregation to combine topology-aware Clifford components through learnable energy- and scale-sensitive weighting for modality fusion.AHA is designed as a dynamic filter for task-adaptive fusion representation.
  • LION: LION uses Clifford Geometric Propagation to construct a MAG-specific geometric manifold, model topology as rotations, and enable modality-aware high-order interaction for alignment.Its geometric potential regulates neighboring signals during transport.
  • Results: 5.24% and 7.68% average improvements over SOTA baselines are reported for graph and modality tasks, respectively, across evaluations from 6 domains.The introduction reports these averages as evidence of LION’s performance advantage.

III. METHODOLOGY

LION represents topology and modalities in a shared Clifford geometric manifold, then propagates and aggregates the resulting components. CGP performs topology-aware modality interaction and AHA uses component energy and propagation scale for adaptive fusion.

  • Clifford Algebra and Geometric Manifold: Clifford algebra models each MAG node in a local tangent space, with edges as geodesic connections and geometric products unifying topology rotations with modality basis vectors.This creates a modality-aware high-dimensional geometric manifold.
  • Clifford Algebra and Geometric Manifold: The geometric product decomposes interactions into symmetric inner products for intra-modality compatibility and antisymmetric outer products for cross-modality topology curvature.The outer product forms a bi-vector plane associated with cross-modality interaction.
  • Clifford Geometric Propagation: CGP aligns modalities by parallel-transporting orthogonal geometric basis vectors along topology curvature using a spatial rotor and geometric potential.The rotor determines geometric transport, while the potential controls neighboring contributions.
  • Initialization: Modality-specific projections map unequal raw dimensions into a shared feature dimension before normalized features are assigned to orthogonal Grade-1 Clifford basis vectors.The formulation supports K modalities algebraically, although experiments use the two-modality text-image setting.
  • Clifford Geometric Propagation: CGP computes feature-wise semantic compatibility and oriented cross-modality interaction statistics, converts them into bounded propagation weights, and applies normalized Clifford transport.The stabilized rotor approaches the identity as the bi-vector norm approaches zero.
  • Theoretical Analysis: The theoretical analysis establishes a stability bound for the Clifford construction and a conditional spectral contraction result for a fixed normalized CGP operator.The contraction requires symmetric potentials, reciprocal isometric transports, fixed edge operators, and spectral radius below one on the complementary subspace.

C. Adaptive Holographic Aggregation

Adaptive Holographic Aggregation filters propagated Clifford components by energy and reconciles their representations across propagation scales for task-adaptive modality fusion. Its stability analysis bounds both grade-gating changes and deviations from learned multiscale consensus.

  • AHA adaptively filters propagated Clifford components according to their energy and reconciles representations across propagation scales.
  • Energy-aware Component Filtering: The propagated state remains Grade-1, while each modality-axis component reflects scalar and bi-vector interaction statistics used to construct CGP.
  • Energy-aware Component Filtering: AHA uses learnable gating to attenuate less informative topology-aware modality components without deterministically removing them.
  • Scale-aware Resonance Fusion: AHA reconciles topology scales by weighting propagated representations according to their learned consensus profile across propagation depths.
  • Theoretical bounds separately constrain representation changes from grade gating and deviations from learned multiscale consensus without assuming downstream optimality.

D. Algorithm and Complexity Analysis

LION separates Clifford-based representation construction from task-specific optimization through CGP followed by AHA. Its cached propagation reduces repeated structural computation while supporting multiple graph and modality task heads.

  • Algorithm: LION transforms raw multimodal inputs through two decoupled phases: CGP for modality alignment and AHA for modality fusion.
  • Experimental Datasets: Table I summarizes applicable task types, while evaluated dataset-task pairs are specified by the corresponding result-table headers.
  • Algorithm: Propagated Clifford tokens can be shared by node-, edge-, and modality-level heads, supporting heterogeneous MAG tasks.
  • Complexity Analysis: The one-time CGP preprocessing costs O(M ·2K ·D+L·M ·D), while trainable AHA costs O(L · N · D) per epoch.
  • Complexity Analysis: Cached multiscale representations require O(L·N·D) memory, and learnable projection and gating parameters occupy O(D2) space.

IV. EXPERIMENTS

The experiments evaluate LION through setup and algorithmic procedures covering modality alignment and fusion. The evaluation framework addresses effectiveness, ablation, interpretability, robustness, and scalability.

  • Evaluation Questions: The experiments define five questions covering effectiveness, ablation, interpretability, robustness under sparsity, and scalability.
  • Algorithm: Algorithm 1 takes a multimodal-attributed graph and model-layer depth as inputs and produces learned node- or modality-level representations.
  • Clifford Geometric Propagation: CGP lifts Euclidean modality features into the Clifford manifold, constructs geometric interactions, and performs curvature-adaptive high-order propagation.
  • Adaptive Holographic Aggregation: AHA quantifies information density, gates propagated interaction channels, and integrates multiscale receptive fields through consensus-based weighting.

A. Experimental Setup

LION is evaluated on diverse multimodal-attributed graph benchmarks, task protocols, and baselines, with results spanning graph prediction, modality retrieval, and generation. Its performance and ablations support the proposed alignment and fusion design.

  • Datasets: Nine MAG datasets across six domains cover graph-centric and modality-oriented tasks at scales ranging to more than 0.68M nodes and 7.23M edges.The benchmarks include supervised, unsupervised, retrieval, and generation settings.
  • Evaluation Protocols: Evaluation includes node classification, link prediction, node clustering, modality retrieval, Graph-to-Text, and Graph-to-Image under stated benchmark protocols.Graph-task and ablation results report mean±std over five runs, while some retrieval and generation results follow single-run protocols.
  • Graph Tasks: LION consistently outperforms baselines across graph datasets, tasks, and metrics, exceeding leading NTSFormer and UniGraph2 by an average 5.84% in node classification.The reported gains extend across classification and link-level prediction rather than a single supervised objective.
  • Modality Tasks: LION achieves SOTA performance across modality-task metrics, including SemArt G2Image gains of 4.76% in CLIP-S and 5.18% in DINOv2-S.The results cover retrieval and generation.
  • Ablation Study: Removing either CGP component reduces performance, while omitting AHA energy, scale, or consensus mechanisms also degrades results.The ablations support geometric rotation, geometric potential, component-wise filtering, and adaptive fusion as necessary design elements.

D. In-depth Analysis

The analysis interprets CGP and AHA through visualizations, sparsity tests, and efficiency studies. Results indicate adaptive alignment, depth-dependent behavior, robustness to missing data, and rapid convergence.

  • CGP Module: CGP facilitates intra- and inter-modality interactions through modality-aware geometric manifolds and topology-adaptive high-order propagation.
  • CGP Module: After CGP, modality representations cross prior semantic boundaries while gathering related modalities and improving modality-specific compactness.
  • CGP Module: Smaller CGP depth suits dense graphs, whereas sparse graphs benefit from greater depth to obtain sufficient receptive fields.
  • AHA Module: AHA filters modality channels through energy-based adaptive gating and integrates multi-scale structural contexts according to propagation depth.
  • Robustness Analysis: LION remains robust across sparsity scenarios and outperforms other methods across downstream tasks, using high-order propagation and channel filtering to handle incomplete evidence.
  • Efficiency Analysis: LION converges rapidly and reaches peak performance in significantly fewer epochs, while cached CGP computations and decoupled training improve efficiency.
  • Conclusion: The paper reports bounded geometric sensitivity, conditional spectral contraction, AHA stability bounds, and superiority over SOTA baselines across graph and modality tasks.

APPENDIX A PROOF OF GEOMETRIC STABILITY IN THEOREM 1

The appendix proves geometric stability for the CGP construction and contraction for its normalized cached propagation operator under explicit assumptions.

  • Geometric Stability: Positive stabilizers ensure rotor and potential continuity even as the bi-vector norm approaches zero.
  • Geometric Stability: Theorem 1 stability follows from Lipschitz encoder, normalization, Clifford lift, rotor, potential, and bounded-degree propagation components.
  • Spectral Contraction: The normalized CGP update is analyzed for undirected graphs with symmetric potentials, reciprocal isometric transports, fixed edge operators, and self-loops.
  • Spectral Contraction: The connection Laplacian is positive semidefinite, and degree normalization makes the propagation operator self-adjoint under the degree-weighted inner product.
  • Spectral Contraction: If the complementary invariant subspace has spectral radius ρ < 1, repeated propagation contracts toward the unit-eigenspace component.
  • Scope: The contraction rate applies to the fixed reciprocal transport operator used for cached CGP, not arbitrary time-varying edge operators.

APPENDIX C PROOF OF AHA STABILITY AND CONSENSUS BOUND IN

The appendix establishes AHA stability and a consensus-deviation bound using the affine Clifford projection and soft component gating. These results do not assume downstream-optimal consensus representations.

  • AHA Stability: The AHA stability proof uses only the linear part of the affine Clifford projection and does not require a downstream-optimal learned consensus profile.
  • AHA Stability: Subtracting two affine outputs cancels the bias term, enabling an operator-norm and triangle-inequality bound.
  • AHA Stability: The first inequality quantifies representation change introduced by the soft component gate without assuming attenuated components are noise.
  • Consensus Bound: The second inequality bounds consensus deviation but does not identify Hctx_u with the unknown downstream-optimal representation.

APPENDIX D THEORETICAL FOUNDATIONS OF LION

LION’s theoretical foundation combines Clifford geometric propagation with adaptive holographic aggregation, while the appendix establishes scoped stability and consistency properties. Its computational design caches training-free propagation and trains only the aggregation stage.

  • Theoretical properties: Three theorems bound Clifford sensitivity, conditional spectral contraction, and AHA-induced representation change under stated assumptions.These results support stability and operator consistency but do not establish that attenuated grades are noise or that the consensus is optimal.
  • Algorithmic flow: LION initializes a Clifford geometric manifold, models topology through geometric products, and produces aligned curvature-adaptive tokens with potential-gated high-order propagation.The geometric product supplies compatibility and cross-modality discrepancy statistics for topology-aware transport.
  • Algorithmic flow: AHA filters Clifford token components using learnable energy estimates and fuses propagation depths through scale-aware resonance against a consensus profile.The final representation is projected back into Euclidean space after filtering and scale weighting.
  • Complexity: The Clifford multi-vector dimension is D = 2Kd, with manifold initialization costing O(N · D), geometric construction O(M · 2K · D), and propagation O(L · M · D).N and M denote node and edge counts, L is propagation depth, and K is the number of modalities.
  • Complexity: Because CGP is training-free and parameter-independent, it runs once and is cached, while training optimizes AHA with O(L·N ·D) scaling.AHA’s energy-aware filtering and scale-aware resonance fusion are the training-time operations.

APPENDIX F DATASET DETAILS

The appendix documents nine text-image benchmarks spanning graph, retrieval, and generation settings. Dataset graphs use benchmark relations or independent metadata while excluding downstream targets from edge construction.

  • Scope and construction: All nine benchmarks provide text and image attributes, and empirical claims are restricted to the K = 2 text-image setting.The Clifford formulation supports K modality axes, but the reported experiments evaluate only two modalities.
  • Scope and construction: Edges are constructed from benchmark relations or independently available source metadata, excluding labels, held-out links, retrieval correspondences, and generation references.This protocol is intended to avoid target leakage.
  • Modality-task datasets: SemArt is used for Graph-to-Image generation, Flickr30k for Graph-to-Text generation, and Ele-fashion for modality retrieval.These datasets retain paired textual and visual node attributes while testing different modality-oriented capabilities.
  • Graph-task datasets: The graph-task benchmarks include Cloth, Goodreads, Movies, Grocery, RedditS, and Sports, covering link prediction, node classification, and node clustering.Cloth and Sports support link prediction; Goodreads and Movies support node classification; RedditS and Grocery support node clustering.

APPENDIX G BASELINES DETAILS

The baseline appendix describes graph neural, multimodal, graph-transformer, generative, clustering, and foundation-model comparators. Implementations are evaluated under shared splits and metrics while preserving released backbones and preprocessing where available.

  • Graph neural baselines: GCN, GCNII, GAT, and GATv2 represent localized convolution, deep residual propagation, masked attention, and dynamic attention baselines.Their stated designs target topology-aware representation learning, over-smoothing mitigation, or adaptive neighbor weighting.
  • Multimodal baselines: MMGCN and MGAT model multimodal recommendation through modality-specific graphs or gated attention over parallel interaction graphs.Both baselines explicitly address modality importance or high-order connectivity during multimodal preference learning.
  • Graph-conditioned and generative baselines: Graph4MM, InstructG2I, MLaGA, and GraphGPT-O incorporate topology into attention, generation, or language-model alignment mechanisms.Their methods include hop-diffused attention, semantic neighbor prompts, contrastive graph pre-training, and hierarchical Q-Formers.
  • Specialized and foundation baselines: DMGC, DGF, and MIG-GT address hybrid neighborhood structure, feature denoising, or modality-specific propagation with filtering, contrastive, or global-transformer components.NTSFormer handles isolated cold-start classification, while UniGraph2 uses frozen encoders, mixture-of-experts alignment, and a universal GNN.
  • Evaluation protocol: Baselines retain released backbones and preprocessing when available, use the same splits and metrics, and adapt only task heads when required.Validation data determines hyperparameters and early stopping; test data is not used for selection.

APPENDIX H EVALUATION PROTOCOLS

The evaluation covers supervised classification and link prediction, unsupervised clustering, cross-modal retrieval, and graph-conditioned generation. Protocols standardize encoders and dimensions while using task-specific objectives, metrics, and training settings.

  • Graph tasks: Node classification uses cross-entropy-trained embeddings with Accuracy and F1-score, while link prediction ranks candidate edges using MRR and Hits@K.Link prediction aligns structural proximity with cross-modal semantic similarity.
  • Graph tasks: Node clustering evaluates unsupervised semantic grouping with NMI and ARI after disentangling homophilous and heterophilous graph views.The protocol uses reconstruction, contrastive alignment, and clustering losses.
  • Modality tasks: Modality retrieval learns a shared latent space for image-to-text and text-to-image search using contrastive learning with temperature τ = 0.07.Candidates are ranked by pairwise similarity after projection into the unified space.
  • Generation tasks: Graph-to-Image uses semantic PPR neighbor sampling, Graph-QFormer encoding, and conditional latent diffusion with graph classifier-free guidance.The protocol standardizes image resolution at 256 and uses the Stable Diffusion v1.5 backbone.
  • Standardization: Core configurations use CLIP-ViT-L/14, 768-dimensional node embeddings, Adam or AdamW, and five independent runs for graph comparisons and ablations.Retrieval and generation follow their benchmark protocols, and entries without ± are single-run evaluations.
  • Generation tasks: Graph-to-Text generates descriptions from graph-structured multimodal context, evaluated with BLEU-4, ROUGE-L, and CIDEr.These metrics respectively emphasize n-gram fluency, longest-common-subsequence recall, and TF-IDF-weighted caption consensus.
Loading 2608.24795v1…