PaperlayerView exact revision on arXiv

Source-linked AI summary

PANDA: Prior-guided Attentional Dual-path Architecture

Rishabh Jain, Federico Belotti, Stefano Coniglio, Pietro Lio, Stefano Fiorini

arXiv:2610.13635v1cs.LG

TL;DR

HNNs face a trade-off between potentially misleading fixed weights and competitive normalised attention, while directed operators often fix head and tail contributions. PANDA combines structural priors and learned attention with a non-competitive auxiliary path; across eight benchmarks, it averages +5.18 pp improvement. With fixed structural coefficients, it also recovers GeDi-HNN’s complex-valued convolution through two-stage message passing.

  • Problem

    HNNs face a trade-off: fixed weights may mislead in heterophilic settings, normalised attention makes senders compete, and directed operators often fix head and tail contributions.

    Source: §1 · pp. 1–2

  • Method

    PANDA combines a main path interpolating learned receiver-normalised attention and structural priors with a non-competitive auxiliary path for directed and undirected HNNs.

    Source: §1 · p. 2

  • Results

    +5.18 pp average improvement across eight benchmarks with four HNN backbones spanning spatial/spectral and directed/undirected architectures.

    Source: §1 · p. 2

  • Takeaways & Limitations

    With fixed structural coefficients, PANDA recovers GeDi-HNN’s complex-valued convolution through two-stage message passing.

    Source: §1 · p. 2

    When coefficients are learned, PANDA does not necessarily preserve the spectral operator’s Hermitianicity.

    Source: §4.1 · p. 7

Abstract

from arXiv · show

Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior-guided AttentioNal Dual-path Architecture), a plug-in for two-stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver-normalised attention and structural incidence priors, while a non-competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex-valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real-world benchmark datasets. The code is available at https://github.com/jainris/panda.

1 Introduction

PANDA addresses limitations of fixed structural weights and receiver-normalised attention in hypergraph message passing. It combines a prior-guided main path with a non-competitive auxiliary path and improves benchmark performance across diverse HNNs.

1 Introduction

  • Fixed structural coefficients can propagate misleading information in heterophilic hyperedges, while receiver-wise normalisation prevents multiple senders from receiving high weight simultaneously.Directed operators also commonly lock head and tail contributions to fixed coefficients, independent of propagated features.Source: §1 · pp. 1–2
  • PANDA combines a main path interpolating structural incidence priors and learned receiver-normalised attention with a gated, non-competitive auxiliary path.The plug-in applies to two-stage message-passing architectures in both directed and undirected settings.Source: §1 · p. 2
  • With fixed structural coefficients, PANDA recovers GeDi-HNN’s complex Hermitian convolution through two-stage message passing, but not through directed star expansion.The star-expansion realization differs in phase.Source: §1 · p. 2
  • +5.18 pp average gain across eight benchmarks, with gains up to +31.01 pp, across four directed/undirected and spatial/spectral HNN backbones.The largest reported gains are on Telegram.Source: §1 · p. 2
  • PANDA beats a parameter-matched variant with a normalised auxiliary path on 27 of 32 model–dataset pairs, averaging +2.60 pp.The comparison isolates the non-competitive auxiliary path from parameter count and separate transformations.Source: §1 · p. 2
  • The authors separately analyse the auxiliary path’s contribution during training and the trained model’s dependence on it at inference.Its predictive effect is reported as a broad margin improvement, not only neighbour-majority correction.Source: §1 · pp. 2–3

2 Related Work

Related work spans spectral and spatial HNNs, directed hypergraph operators, and attention or gating mechanisms. PANDA is positioned against limitations in directed coverage and competitive attention.

Undirected and Directed Hypergraph Neural Networks

  • Undirected spectral and spatial HNNs use Laplacian-based convolution or local hyperedge aggregation, with many models interpretable through clique or star expansions.PhenomNN combines both expansions.Source: §2 · p. 3
  • Directed hypergraph work is comparatively sparse; GeDi-HNN uses a complex-valued Hermitian operator based on the Generalized Directed Laplacian.DHRL approximates a directed hypergraph Laplacian.Source: §2 · p. 3

Attention in GNNs and HNNs; Gated Attention

  • HCHA and HyperGAT use receiver-wise normalisation, so increasing one sender’s weight reduces the mass available to others.Prior gating mechanisms also accompany competitive normalisation, and the cited methods do not explore hypergraphs.Source: §2 · p. 3

3 PANDA: Prior-guided AttentioNal Dual-path Architecture

PANDA intercepts incidence messages in two-stage node–hyperedge–node architectures. Its main path blends learned attention with structural priors, while its auxiliary path adds separately transformed messages without receiver-wise normalisation.

3 PANDA: Prior-guided AttentioNal Dual-path Architecture

  • PANDA intercepts node-to-hyperedge and hyperedge-to-node messages in a two-stage architecture and reweights the incidence messages.The target architecture supplies the learnable transformation, aggregation, and update functions.Source: §3 · p. 4

3.1 PANDA’s Main Path

  • The main path combines GATv2 attention with fixed structural coefficients, recovering the prior-only routing when its learned gate is zero.The gate can shift toward the prior where it is reliable and retain learned scores where the prior is misleading.Source: §3.1 · pp. 4–5

3.2 Auxiliary Path

  • The auxiliary path adds separately transformed messages with locally complement-bounded coefficients that are not normalised across a receiver’s receptive field.Thus, messages with small main-path coefficients can still contribute after transformation; the stated gate bounds assume structural priors are scaled so main coefficients lie in [0, 1].Source: §3.2 · p. 6

4 Generalised Directed Spectral Convolution as Two-Stage Message Passing

The paper addresses whether directed complex-Hermitian convolution can be expressed as two-stage message passing. It establishes such a realization for GeDi-HNN but distinguishes it from propagation on a directed star expansion.

4 Generalised Directed Spectral Convolution as Two-Stage Message Passing

  • Prior work places many undirected HNNs in a two-stage message-passing framework, but the directed complex-Hermitian case is not immediate.The paper studies this connection to support applying PANDA to GeDi-HNN.Source: §4 · p. 6

4.1 A complex-valued message-passing scheme

  • PANDA with additive aggregations recovers GeDi-HNN’s convolution through complex-valued two-stage message passing using the incidence matrix and its conjugate.The construction relies on the same complex incidence matrix and conjugate used to define GeDi’s Laplacian.Source: §4 · p. 6, §4.1 · p. 7
  • Directed star-expansion propagation generally differs from the original operator in phase, and learned coefficients do not necessarily preserve Hermitianicity.The Hermitian property holds in the frozen-prior case.Source: §4.1 · p. 7

5 Experimental Evaluation

PANDA is evaluated as a plug-in across four hypergraph backbones and eight node-classification datasets. Its strongest gains occur on heterophilic benchmarks, while ablations and auxiliary-path analyses examine how its components contribute.

5.1 Experimental Setup

  • The evaluation compares PANDA with the original formulation across four spectral or spatial, directed or undirected HNN backbones, using eight real-world datasets and ten-run test-accuracy averages.The datasets use a 50%/25%/25% training, validation, and test split.Source: §5.1 · p. 7

5.2 Results

  • PANDA improves accuracy by an average of 5.18 percentage points over VANILLA across eight datasets, with gains concentrated on heterophilic benchmarks and little change on homophilic citation datasets.The largest reported gain is 31.01 percentage points on Telegram with PhenomNN.Source: Table 1 · p. 8, §5.2 · p. 8
  • Attention-only ablations do not consistently recover VANILLA accuracy; adding the structural prior restores much of it, while the full auxiliary path helps in cases where the main path remains limited.PANDA also consistently outperforms the parameter-matched Norm Aux variant, with a few small exceptions.Source: §5.2 · pp. 8–10, Table 4 · p. 10
  • PANDA also improves GCN across datasets and consistently outperforms GAT, except for small differences on homophilic datasets and a larger difference on email-Enron.The comparison covers GCN, GAT, and PANDA on GCN.Source: §5.2 · p. 9, Table 2 · p. 9
  • Disabling both auxiliary gates sharply reduces accuracy on Telegram and Roman-empire, and the auxiliary path shifts margins against misleading neighbour-majority signals on both datasets.Strict neighbour-majority rescues explain only part of the helped flips, indicating that the margin gains extend beyond that class comparison.Source: §5.2 · p. 9
  • The auxiliary representation is approximately orthogonal to the main representation in the hyperedge-to-node pass; further analyses report parameter-efficiency and inference-speed improvements in several settings.The geometric evidence supports a distinct message channel but does not by itself explain the margin gains.Source: §5.2 · pp. 9–10

6 Conclusions

PANDA combines structural priors and receiver-normalised attention with a non-competitive auxiliary path. Across four HNNs, it improves heterophilic benchmark results without degrading homophilic ones, but remains limited to two-stage message-passing HNNs.

6 Conclusions

  • Across four HNNs, PANDA improves heterophilic benchmarks without degrading homophilic ones, acting as a corrective plug-in when structural propagation is misaligned with the data.The auxiliary branch is informative to trained models and shifts predictions away from misleading neighbourhood signals on heterophilic datasets.Source: §6 · p. 10
  • PANDA’s main path interpolates between receiver-normalised attention and structural incidence priors, while its non-competitive auxiliary path supplies information the main path cannot express alone.With fixed structural coefficients, the paper also shows a two-stage realisation of GeDi-HNN’s complex Hermitian convolution, but not equivalence to a Hermitian graph operator after directed star expansion.Source: §6 · p. 10
  • PANDA is restricted to two-stage message-passing HNNs; extending it to broader propagation paradigms, including sheaf neural networks, is future work.Source: §6 · p. 10

A Proof of our theoretical results

The proof establishes that PANDA can recover the GeDi convolution under fixed complex structural coefficients, while ordinary directed star expansion does not reproduce the hypergraph operator. The distinction follows from the cross-role phases in the directed setting.

A Proof of our theoretical results

  • With fixed complex coefficients and the auxiliary path switched off, PANDA’s two-stage aggregation recovers the GeDi convolutional layer.The equivalence uses the conjugate incidence B* from nodes to hyperedges, followed by B back to nodes.Source: A Proof of our theoretical results · pp. 14–16
  • Neither one GeDi step nor two-hop propagation on the directed star expansion equals the original hypergraph operator Q_N.The two-hop cross-role phases differ from those encoded by the hypergraph operator.Source: A Proof of our theoretical results · p. 15
  • In the two-vertex, one-edge example, the star-expanded two-step coefficient differs in phase from the hypergraph coefficient i/2.The example uses one tail node, one head node, and a unit-weight directed hyperedge.Source: A Proof of our theoretical results · p. 16

B Experimental Setup

The experimental setup uses public baseline implementations, two workstations, and a broad hyperparameter search for the backbones and PANDA variants. Settings are evaluated over ten dataset splits, with the initial hyperedge representations derived from vanilla backbone scores.

B.1 Code Repository

  • Baseline models come from public GeDi-HNN implementations, while the research code was to be distributed under Apache 2.0 upon acceptance.The paper says the code is attached as supplementary material.Source: §B.1 · p. 16

B.2 Hardware Details

  • The experiments use two workstations, and each hyperparameter setting is trained and validated over ten dataset splits.For GeDi+PANDA, a setting takes 14.2 minutes on Telegram and 40.4 minutes on Roman-empire on average.Source: §B.2 · p. 16

B.3 Hyperparameter tuning

  • Models are trained for up to 500 epochs with early stopping at 200, using Adam and searched learning-rate, weight-decay, and dropout ranges.The tuning also varies backbone-specific architecture settings and uses a two-layer final classifier when selectable.Source: §B.3 · p. 17
  • PANDA and its ablations add attention-head and source–destination-weight choices to each backbone’s hyperparameter search.The search also varies attention version, with up to eight heads.Source: §B.3 · p. 17
  • Initial hyperedge representations are computed using the backbone’s vanilla scores because the datasets provide no initial hyperedge representations.The same strategy is used in subsequent layers.Source: §B.3 · p. 17

B.4 Dataset Descriptions

The experiments use eight real-world datasets and report dataset statistics spanning graph size and homophily measures. The computational comparison reports parameter and inference-time outcomes for PANDA variants against vanilla models.

B.4 Dataset Descriptions

  • The eight real-world datasets include citation, email, WebKB, Telegram, and Wikipedia-based node-classification settings.The dataset descriptions identify different node and edge meanings across these settings.Source: §B.4 · p. 17
  • Table 5 reports dataset size and structure measures, including node and hyperedge counts, average sizes, and several homophily measures.The homophily measures include clique-expansion, node, and hyperedge homophily.Source: Table 5 · p. 18

C Computational and Parameter Comparison

D Results on Additional Datasets

On additional datasets, PANDA improves performance across the backbones on Questions, with especially large HyperGCN gains on Wisconsin and email-EU. The embedding-stability analysis finds no disproportionate norm growth, although norm changes vary by dataset.

D Results on Additional Datasets

  • The additional-dataset evaluation includes Wisconsin, email-EU, and Questions, with Questions serving as a scalability test at 48,921 nodes.The main table reports eight datasets because of space constraints; these three are reported additionally.Source: D Results on Additional Datasets · p. 18
  • PANDA improves performance across all backbones on Questions; on Wisconsin and email-EU, HyperGCN benefits massively while other backbones show small positive gains or nearly match vanilla.Table 8 reports mean accuracy for Wisconsin and email-EU and percentage AUC-ROC for Questions.Source: D Results on Additional Datasets · pp. 18–19, Table 8 · p. 19
  • On Questions, PANDA uses fewer parameters across backbones and is more than 2× faster at inference except for HyperGCN, which is 1.4× slower.Table 9 reports parameter counts and mean wall-clock inference times.Source: D Results on Additional Datasets · p. 19, Table 9 · p. 19

E Embedding Stability and Degree Sensitivity

  • Across datasets, PANDA produces comparable, larger, or smaller embedding norms than GeDi, with no clear trend of increased instability.The reported norm growth is larger on Telegram, Pubmed, and email-Enron, and smaller on Texas and Squirrel.Source: E Embedding Stability and Degree… · p. 19, Table 10 · p. 20
  • PANDA does not cause disproportionate node-embedding norm growth; on almost half the datasets its coefficient of variation is lower than vanilla’s.Table 11 uses coefficient of variation to describe how unevenly node norms grow.Source: E Embedding Stability and Degree… · p. 20, Table 11 · p. 20

F Extended analysis

The extended analysis evaluates GeDi-PANDA routing on five datasets using ten saved checkpoints per dataset. It reports routing quantities alongside test accuracy and specifies how checkpoint and layer variation affect the reported statistics.

F Extended analysis

  • The routing analysis covers Citeseer, Cornell, Roman-empire, Telegram, and Texas, evaluating ten saved checkpoints per dataset in inference mode.Accuracy and routing statistics are reported as means and standard deviations across checkpoints.Source: F Extended analysis · p. 20
  • Roman-empire’s routing standard deviations pool layer–checkpoint instances, so they are not directly comparable to single-layer dataset standard deviations.The analysis states layerwise behavior explicitly when pooling obscures substantial layer structure.Source: F Extended analysis · p. 20
  • The analysis defines routing variables for combining learned attention, the structural prior, and the auxiliary path; auxiliary quantities are measured at test time with both auxiliary gates set to zero in the aux-off baseline.This baseline is used to calculate auxiliary-path quantities.Source: F Extended analysis · p. 20

F.1 Routing statistics and gate interventions

The routing analysis describes how PANDA combines attention, structural priors, and auxiliary messages across datasets. Inference-time gate interventions test whether the auxiliary path’s measured magnitude corresponds to predictive dependence.

F.1 Routing statistics and gate interventions

  • Routing statistics summarize attention uniformity, prior-gate mass, auxiliary gates, and auxiliary coefficients across stages and head/tail roles.Uniformity is computed over non-singleton receiver groups; role-specific table values are computed by masking incidences for analysis.Source: §F.1 · pp. 20–21, Table 13 · p. 22
  • The analysis compares main and auxiliary message magnitudes using ratios and shares, but these magnitude statistics alone do not establish the auxiliary path’s functional role.The authors use inference-time intervention to test predictive dependence.Source: §F.1 · pp. 22–23
  • Telegram’s V→E learned attention is uniform and its prior mass is fixed at 0.500, so non-uniform main-path routing must come from the structural prior.The authors attribute the attention degeneracy to Telegram’s uninformative one-dimensional node features, not to the interpolation mechanism generally.Source: §F.1 · pp. 22–23
  • On Telegram, auxiliary messages account for 96.4% of head-side and 86.4% of tail-side V→E message magnitude, far exceeding the main-path norm.The corresponding auxiliary-to-main norm ratios are 27.578 and 6.423.Source: §F.1 · p. 23
  • Roman-empire uses the auxiliary path substantially, with 42–48% of message norm across stages and roles; Citeseer, Cornell, and Texas have smaller auxiliary shares.On those latter three datasets, the auxiliary path is active but does not dominate main-path norm on average.Source: §F.1 · p. 23
  • Closing the auxiliary gates causes large accuracy drops on Telegram and Roman-empire, while mean effects are much smaller on Citeseer, Cornell, and Texas.The reported drops when both gates are closed are 54.0 percentage points on Telegram and 27.7 on Roman-empire.Source: §F.1 · p. 23, Table 15 · p. 24
  • Inference-time gate interventions measure the trained model’s dependence on the auxiliary path, rather than its marginal contribution during training.For Roman-empire, leave-one-out interventions disable both gates at one layer, while cumulative interventions disable them jointly across layers.Source: §F.1 · pp. 23–24
  • Roman-empire’s auxiliary-path predictive role is spread across all four layers, with the first layer having the largest leave-one-out accuracy drop at 7.4 percentage points.The all-off drop is 27.7 points, exceeding the 21.6-point sum of the four individual leave-one-out drops, indicating non-additive intervention effects.Source: Table 16 · p. 24, §F.1 · p. 25

F.2 Auxiliary Path Analysis

Inference-time interventions show that the auxiliary path’s predictive role varies by dataset and is not limited to correcting misleading neighbour-majority predictions. The analysis also examines broader true-class margin shifts and distinguishes rescue from reinforcement.

F.2 Auxiliary Path Analysis

  • On low-homophily Roman-empire and Telegram, the auxiliary path increases the true-versus-neighbour-majority margin by 5.29 and 3.58, respectively, acting as a counter-neighbourhood signal.The neighbour-majority is misleading for 95.7% of Roman-empire nodes and 81.7% of Telegram nodes.Source: §F.2 · p. 26
  • Strict neighbour-majority rescue accounts for only 2.20% of Roman-empire helped flips and 16.35% on Telegram, versus helped-flip rates of 29.44% and 56.54%.The auxiliary path’s predictive effect therefore extends beyond correcting predictions that follow the misleading neighbour-majority class.Source: §F.2 · p. 27
  • The auxiliary path’s neighbour-majority effect is not universal: Citeseer shows a small intervention effect and a slight shift toward its usually informative neighbour-majority, while Cornell and Texas show near-zero shifts.The authors describe Cornell and Texas as intermediate in clique-expansion homophily and intervention effect.Source: §F.2 · p. 27
  • The analysis also measures true-class margin shifts against each node’s strongest non-true class, separating this broader comparison from the neighbour-majority axis.The competitor is chosen independently of the neighbour-majority class and separately in full and aux-off logits.Source: §F.2 · p. 27
  • On Roman-empire, the auxiliary path both reinforces correct predictions and rescues wrong ones, but rescue dominates: 1,668 of 2,182 aux-off-wrong nodes flip to correct.Mean margin shift is +13.97 on aux-off-wrong nodes and +2.99 on aux-off-correct nodes.Source: §F.2 · p. 28
  • On Telegram, rescue is nearly the only observed mechanism: 29.4 of 34 aux-off-wrong nodes per checkpoint flip to correct, while only 18 aux-off-correct nodes are available.The aux-off-wrong subset has a mean margin shift of +9.39.Source: §F.2 · p. 28
  • Citeseer shows no consistent rescue signal: only 15.2 of 210 aux-off-wrong nodes flip to correct, consistent with its small intervention effect.The mean margin shift is +0.09 and only 41.9% of those nodes have a positive shift.Source: §F.2 · p. 28
  • Rescue percentages for Cornell and Texas are not statistically meaningful in this analysis because each checkpoint has only about 11 and 6 aux-off-wrong nodes, respectively.A single misclassified node changes the rate by roughly 10–17 percentage points.Source: §F.2 · p. 28

F.3 Pass-wise geometry of the auxiliary path

This section examines pass-wise auxiliary-path geometry by comparing source representations with destination-aggregated contributions, and records checklist disclosures about anonymized supplementary code, human-subject research, and core-method LLM use.

F.3 Pass-wise geometry of the auxiliary path

  • Source-side cosine measures alignment before message passing, whereas receiver-side cosine measures the aggregated contributions after attention-weighted summation, including incidence weights and cross-source interference.The results support a distinct auxiliary feature direction, especially in E→V, where aggregation generally tightens near-orthogonality; V→E alignment weakens in Telegram and deeper Roman-empire layers.Source: §F.3 · pp. 28, 30, 32
  • 0.38–0.55 source-side cosine on Citeseer, Cornell, Texas, and Roman-empire layer 0 shows V→E alignment, while receiver-side aggregation weakens but does not invert it.For example, receiver-side cosine falls from 0.55 to 0.19 on Cornell and from 0.49 to 0.21 on Texas. Telegram is lower at +0.138, and Roman-empire declines from +0.529 at layer 0 to +0.146 at layer 3, settings where the auxiliary path is most predictively important.Source: §F.3 · p. 29
  • −0.05 to +0.10 receiver-side cosine on Citeseer, Cornell, Texas, and Telegram shows E→V contributions remain nearly orthogonal after aggregation.On Roman-empire, source-side cosine shifts from +0.125 at layer 0 to −0.160 and −0.082 in layers 2 and 3; receiver-side values are +0.006 at layer 0 and −0.087 at layer 2.Source: §F.3 · pp. 29–30

NeurIPS Paper Checklist

  • The checklist reports that the paper provides model, dataset, baseline, and tuning details, code with dataset-download scripts, and standard deviations for its results.The authors also direct readers to Appendix B for computing-resource information; the datasets are publicly available.Source: NeurIPS Paper Checklist · pp. 34–36
  • The checklist states that the research conforms to the NeurIPS Code of Ethics and marks societal-impact discussion as not applicable.The authors explain that the work aims to advance machine learning and that they do not feel its potential societal consequences require specific highlighting here.Source: NeurIPS Paper Checklist · pp. 36–37
  • The authors answer yes that new assets are documented and state that anonymized code is provided as supplementary material.The checklist’s asset documentation response is affirmative.Source: NeurIPS Paper Checklist · p. 38
  • The paper marks crowdsourcing and human-subject research, including IRB review, as not applicable because it involves no such research.The checklist also marks its LLM-use declaration as not applicable, stating that core method development does not involve LLMs.Source: NeurIPS Paper Checklist · pp. 38–39

Related papers

Loading 2610.13635v1…