PaperlayerView exact revision on arXiv

Source-linked AI summary

Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagation

Ping Xiong, Shanglin Li, Yi Ding, Thomas Schnake, Shinichi Nakajima

arXiv:2609.35128v1cs.LG

TL;DR

Hyperbolic operations complicate feature attribution because equivalent internal computations can produce different explanations, and local relevance conservation alone does not guarantee consistency. The paper proposes geometry-aware relevance rules and criteria for these cases. LRP-radial-all achieves competitive attribution fidelity across the evaluated datasets, with runtime comparable to Gradient×Input and substantially lower than Integrated Gradients.

  • Problem

    Hyperbolic feature attribution remains less understood than attribution for Euclidean networks, and equivalent geometric computations can yield different explanations despite local relevance conservation.

    Source: §1 · p. 1, §3.2 · p. 4

  • Method

    The paper proposes LRP-radial-all for origin-centered hyperbolic modules, treating geometric scaling as modulation and assigning relevance to the signal branch.

    Source: §1 · p. 2, §4 · p. 5

  • Results

    LRP-radial-all achieves competitive attribution fidelity across MNIST, sEEG, and CIFAR-10, with runtime comparable to Gradient×Input and substantially lower than Integrated Gradients.

    Source: §6 · p. 9

  • Takeaways & Limitations

    The results distinguish local relevance conservation from consistency across equivalent computations and support geometry-aware propagation rules for the specified constructions.

    Source: §1 · p. 2, §6 · p. 9

    The theoretical guarantees apply to specified module realizations and propagation conventions; invariance under alternative treatments of time-coordinate relevance remains open.

    Source: §6 · p. 10

Abstract

from arXiv · show

Hyperbolic neural networks introduce geometric operations that require explicit treatment in relevance propagation. Equivalent geometric realizations can produce different feature attributions, even when local relevance is conserved. We study this problem through Geometric Representation Invariance (GRI), a specialization of Implementation Invariance, and zero-curvature consistency, which requires identity relevance propagation when a geometric module approaches the identity. We propose LRP-radial-all for origin-centered radial modules, treating geometric scaling as modulation and assigning relevance entirely to the signal branch. The rule conserves relevance, is invariant to equivalent radial factorizations, and satisfies zero-curvature consistency, yielding GRI for a specified Poincar\'e-Lorentz logarithmic-map construction. In contrast, a conservative LRP-half baseline can violate both consistency criteria. Experiments on hyperbolic MNIST, sEEG, and CIFAR-10 classifiers assess attribution fidelity, qualitative explanations, and runtime. LRP-radial-all achieves competitive attribution fidelity across datasets with runtime comparable to Gradient$\times$Input and substantially lower than Integrated Gradients. These findings motivate geometry-aware propagation rules that distinguish relevance conservation from consistency across equivalent computations.

1 INTRODUCTION

The paper addresses how feature attribution should handle hyperbolic operations, where equivalent internal computations can yield different explanations. It proposes propagation rules and consistency criteria, then evaluates them on hyperbolic classifiers.

1 INTRODUCTION

  • Hyperbolic neural networks are motivated by their fit to hierarchical structures, whose exponential growth contrasts with Euclidean space’s polynomial volume growth.The introduction notes applications across computer vision, language, foundation models, and biosignal analysis.Source: §1 · p. 1
  • Hyperbolic geometric maps can couple feature coordinates, and equivalent representations can expose different factorizations, challenging consistent attribution.The paper uses LRP’s modular backward rules to study how relevance should pass through these operations.Source: §1 · pp. 1–2
  • LRP-radial-all treats geometric scaling as modulation and assigns relevance to the signal branch in origin-centered hyperbolic modules.The origin supplies a fixed reference point with a Euclidean tangent-space representation.Source: §1 · p. 2
  • The paper specializes Implementation Invariance as GRI and introduces zero-curvature consistency; radial-all satisfies both criteria under the specified construction, while conservative LRP-half can violate them.The stated GRI result applies to the specified logarithmic-map test.Source: §1 · p. 2
  • The evaluation covers attribution fidelity and runtime on hyperbolic MNIST, sEEG, and CIFAR-10 classifiers, alongside controlled consistency tests.The introduction identifies these as evaluation settings, without reporting their outcomes here.Source: §1 · p. 2

2 RELATED WORK

Prior explainability work has begun to account for non-Euclidean geometry, but much of it addresses data-manifold geometry rather than internal hyperbolic operations. The paper positions relevance propagation through those internal operations as a complementary problem.

2 RELATED WORK

  • Hyperbolic architectures have expanded from Poincaré-ball layers to broader fully connected, convolutional, and attention designs, including applications to sEEG.These models use geometric operations such as exponential and logarithmic maps and manifold-valued transformations.Source: §2 · p. 2
  • Manifold Integrated Gradients uses geodesic paths on a learned Riemannian data manifold, while diffeomorphic counterfactual methods use coordinate systems for semantically meaningful input changes.These approaches focus primarily on data-manifold geometry rather than propagation through HNN internal operations.Source: §2 · pp. 2–3
  • Existing LRP approaches differ in how they allocate relevance in multiplicative interactions, including assigning it to the signal branch, splitting it between branches, or uniformly redistributing it.The paper draws on these alternatives to motivate distinct treatments of hyperbolic radial maps.Source: §2 · p. 3

3 GEOMETRY-AWARE CRITERIA

The section introduces hyperbolic network operations and criteria for consistent explanations across equivalent representations and as curvature vanishes. It describes a controlled logarithmic-map comparison and defines zero-curvature consistency.

3.1 HYPERBOLIC NEURAL NETWORKS

  • A representative Poincaré-ball HNN maps an input into the ball, applies hyperbolic linear transformations, bias addition, and tangent-space activation, then maps back for a tangent-space readout.The Lorentz model is presented as a corresponding representation of the same geometry.Source: §3.1 · p. 3

3.2 GEOMETRIC REPRESENTATION INVARIANCE

  • The Poincaré ball and Lorentz hyperboloid are isometric representations, so consistently transformed computations can implement the same input-output function.GRI requires a fixed explanation procedure to give identical input explanations for functionally equivalent models related by the specified isometries.Source: §3.2 · p. 3
  • For propagation-based methods such as LRP, functional equivalence does not ensure GRI because local redistribution depends on intermediate computations and propagation rules.The controlled test compares direct Poincaré logarithmic mapping with a path through Lorentz conversion and its aligned logarithmic map, followed by the same downstream computation.Source: §3.2 · p. 4
  • The controlled GRI test holds the input, prediction, and surrounding computation fixed, then compares relevance propagated back through the two equivalent logarithmic-map paths.The shared downstream computation gives both paths the same relevance before their geometric operations are traversed backward.Source: §3.2 · p. 4

3.3 ZERO-CURVATURE CONSISTENCY

  • Zero-curvature consistency requires relevance propagation through a geometric module to approach identity propagation when the module approaches a fixed coordinate scaling as curvature vanishes.The definition applies for every input and incoming relevance vector.Source: §3.3 · p. 4
  • In the stated coordinate convention, origin Poincaré maps approach identity, while aligned Lorentz spatial logarithmic and exponential maps approach scalings by 1/2 and 2.These constant factors are treated as fixed modulation and receive no separate relevance.Source: §3.3 · p. 4

4 RELEVANCE PROPAGATION RULES

The section defines relevance rules for origin-centered hyperbolic modules, treating geometric scaling as modulation of a signal. LRP-radial-all preserves relevance across equivalent radial factorizations and zero curvature, while the conservative LRP-half baseline can violate both consistency criteria.

4 RELEVANCE PROPAGATION RULES

  • The section applies these rules to origin-centered hyperbolic modules, using standard Euclidean LRP for tangent-space linear layers and a separate signal-only convention for Möbius bias addition.Source: §4 · p. 5

4.1 CONSISTENT RELEVANCE PROPAGATION THROUGH GEOMETRIC MODULATION

  • LRP-radial-all assigns all incoming relevance to the designated signal and none to the geometric modulation factor.The rule is motivated by preserving feature-wise relevance across modulation and by identity propagation in the zero-curvature limit.Source: §4.1 · p. 5
  • For equivalent radial realizations preserving signal dimension and coordinate order, radial-all yields identical input relevance regardless of modulation-factor grouping and conserves signed total relevance.For fixed-proportion rules in the specified family, radial factorization invariance permits only η = 0 or η = 1 when d ≥2; requiring zero-curvature consistency selects η = 1.Source: §4.1 · p. 5

4.2 LRP-HALF AS AN ALTERNATIVE

  • The LRP-half comparison covers log- and exp-map factorizations, but the equivalent Poincaré and Lorentz paths can produce coordinate-wise different relevance despite conserving the total.For the logarithmic-map test, the direct Poincaré path has one radial module and the equivalent Lorentz path has two; LRP-half also fails zero-curvature consistency, whereas radial-all has identity backward action for every curvature.Source: §4.2 · p. 6, Figure 2 · p. 6

5 EXPERIMENTS

Across hyperbolic MNIST, sEEG, and CIFAR-10 classifiers, LRP-radial-all achieves competitive attribution fidelity, with dataset-specific differences across fidelity measures. Its runtime is comparable to Gradient×Input and substantially lower than Integrated Gradients.

5 EXPERIMENTS

  • Across MNIST, sEEG, and CIFAR-10, LRP-radial-all achieves competitive attribution fidelity; its runtime is comparable to Gradient×Input and substantially lower than Integrated Gradients.The evaluation combines sparsity-fidelity metrics and qualitative visualizations across the three classifiers.Source: §5 · p. 6

5.2 SEEG

  • LRP-radial-all has the best average Fidelity+ AUC and comparable Fidelity− AUC to the best baseline on sEEG.Results average trial-level AUCs across held-out folds within each subject, then across nine subject means.Source: Table 1 · p. 7, §5.2 · p. 9

5.1 MNIST

  • On MNIST, LRP-radial-all has the highest Fidelity+ AUC and lowest Fidelity− AUC among evaluated methods, with only marginal improvement over Integrated Gradients.Its heatmaps highlight important curves and intersections, while Integrated Gradients heatmaps appear noisier.Source: Table 1 · p. 7, §5.1 · p. 7

5.3 CIFAR-10

  • On CIFAR-10, LRP-radial-all achieves the lowest Fidelity− AUC, but its Fidelity+ AUC falls below Random under the pixel mean-color replacement protocol.The authors characterize this as stronger retention sufficiency but weaker deletion sensitivity under this perturbation protocol.Source: Table 1 · p. 7, §5.3 · p. 9

5.2 SEEG

  • For subject 2, low-load relevance is more localized, while high-load relevance is more distributed across channels, stronger later in time, and class-dependent in polarity.Frequency-space relevance is concentrated mainly in the delta band, with its magnitude differing between load classes.Source: §5.2 · pp. 7–8, Figure 4 · p. 8, Figure 5 · p. 8

5.4 RUNTIME EVALUATION

  • LRP-radial-all incurs substantially lower computational cost than Integrated Gradients and comparable cost to Gradient×Input.Runtime comparisons cover MNIST, sEEG, and CIFAR-10.Source: Table 2 · p. 7, §5.4 · p. 9

6 CONCLUSION

The paper concludes that LRP-radial-all conserves relevance, preserves allocations across radial factorizations, and satisfies zero-curvature consistency, yielding GRI for a specified Poincaré–Lorentz logarithmic-map construction. It also reports competitive attribution fidelity and favorable runtime, while noting limits on the theoretical guarantees and an open question in the controlled exponential-map test.

6 CONCLUSION

  • LRP-radial-all conserves relevance, preserves incoming allocations across radial factorizations, and satisfies zero-curvature consistency, yielding GRI for the specified Poincaré–Lorentz logarithmic-map construction.LRP-half can fail both consistency criteria despite conserving relevance.Source: §6 · p. 9
  • The theoretical guarantees apply only to specified module realizations and propagation conventions; alternative treatments of time-coordinate relevance in the controlled exponential-map test remain open.Future work includes non-origin base points and structured explanations of message-passing paths and higher-order interactions.Source: §6 · p. 10

A POINCAR´E AND LORENTZ REPRESENTATIONS

  • The origin coordinates are given for the Poincaré ball and Lorentz hyperboloid, with a curvature-rescaled Poincaré–Lorentz isometry and inverse.The supplied passage also states that both spatial conversions are radial.Source: A POINCAR´E AND LORENTZ REPRESENTATIONS · p. 13

B MULTIPLICATIVE ATTRIBUTION AND GROUPING INVARIANCE

  • For scalar–signal products, conservation alone does not determine whether relevance should be shared between the geometric factor and signal or assigned only to the signal.The paper studies equal explanatory status for scalar factors alongside the signal-only formulation used by LRP-radial-all.Source: B MULTIPLICATIVE ATTRIBUTION AND… · p. 13

B.1 LOCAL RULES AND GROUPING INVARIANCE

  • Equal splitting is symmetric and conservative but changes factor relevance when the same product is regrouped, so it is not multiplicatively grouping-invariant.For y = abc, the two parenthesizations allocate proportions (1/4, 1/4, 1/2) and (1/2, 1/4, 1/4).Source: §B.1 · p. 13

B.2 CHARACTERIZATION OF SYMMETRIC FACTOR ATTRIBUTION

For continuous symmetric conservative local rules on positive scalar products, the paper characterizes grouping invariance by a logarithmic allocation. It distinguishes this fixed-factor result from the asymmetric signal-versus-modulation convention used for radial geometric modules.

B.2 CHARACTERIZATION OF SYMMETRIC FACTOR ATTRIBUTION

  • The unique continuous symmetric conservative rule that is multiplicatively grouping-invariant allocates relevance in proportion to each factor’s logarithm.The rule assigns Ra = log a / (log a + log b) Ry and Rb = log b / (log a + log b) Ry.Source: §B.2 · p. 14
  • The characterization concerns fixed lists of positive factors and does not require a separate assumption of invariance under factor expansion.The proof derives the rule from grouping invariance and locality, then extends the result using continuity.Source: §B.2 · p. 14

B.3 CONNECTION TO RADIAL ATTRIBUTION

  • The logarithmic formula does not directly cover general neural activations; radial geometric modules instead distinguish a vector signal from its scalar modulation.Their invariance under splitting or merging radial modules differs from regrouping a fixed list of explanatory factors.Source: §B.3 · p. 14

C.1 RULE FAMILY AND REPEATED PROPAGATION

The paper analyzes fixed-proportion relevance rules for radial modules, then tests their consistency across equivalent geometric computations. It identifies radial-all as the unique rule in that family satisfying both radial factorization invariance and zero-curvature consistency, and contrasts it with LRP-half.

C.1 RULE FAMILY AND REPEATED PROPAGATION

  • Within the fixed-proportion family, relevance is split between the signal branch and a scalar branch redistributed according to squared input coordinates.Across nonzero radial chains in dimension d ≥ 2, factorization invariance holds only for η = 0 or η = 1; both choices conserve relevance.Source: §C.1 · pp. 14–15

C.2 ADDING ZERO-CURVATURE CONSISTENCY

  • Adding zero-curvature consistency selects LRP-radial-all as the unique rule in this family satisfying both criteria for the specified origin Poincaré maps.The η = 0 choice redistributes relevance rather than propagating it identically in the zero-curvature limit.Source: §C.2 · p. 15

D.1 POINCAR´E BRANCH / D.2 LORENTZ BRANCH AND TANGENT ALIGNMENT

  • The origin Poincaré logarithmic map has a radial scalar–signal form, and its aligned Lorentz counterpart is also radial in spatial coordinates.The Lorentz spatial tangent component is divided by two to recover common coordinates.Source: §D.1 · p. 15, §D.2 · p. 16

D.3 FORWARD EQUIVALENCE

  • The direct Poincaré and aligned Lorentz logarithmic-map paths produce identical common tangent coordinates.The equivalence follows from the radial factors under the specified coordinate conversion.Source: §D.3 · p. 16

D.4 RELEVANCE EQUALITY IN THE CONTROLLED TEST

  • Under the specified shared setup, LRP-radial-all yields GRI for the logarithmic-map test by propagating relevance unchanged through each spatial radial module.The test treats the Lorentz time coordinate as a dependent geometric quantity within these modules.Source: §D.4 · p. 16

E.1 POINCAR´E EXPONENTIAL MAP / E.2 LORENTZ EXPONENTIAL MAP AND ALIGNMENT / E.3 RELEVANCE RULES FOR THE SPATIAL RADIAL MODULES

  • The Poincaré and aligned Lorentz exponential maps produce corresponding geometric points when their tangent inputs are aligned.Both spatial outputs have a scalar–signal form, with the constant tangent alignment included in the Lorentz scalar factor.Source: §E.2 · p. 17, §E.3 · p. 17

E.4 A CONTROLLED EXPONENTIAL-MAP TEST

  • In the controlled exponential-map test, LRP-radial-all gives identical relevance at the common input across the direct Lorentz path and the Poincaré-plus-conversion path.The equality extends to the original input when the tangent coordinates come from a shared encoder and its propagation is identical.Source: §E.4 · pp. 17–18

F LRP-HALF: CONSERVATION AND CONSISTENCY COUNTEREXAMPLES / F.1 BRANCH SPLIT, SQUARED-NORM REDISTRIBUTION, AND CONSERVATION

  • LRP-half conserves signed total relevance but can violate GRI and zero-curvature consistency; its squared-coordinate redistribution requires a separate convention at zero input.At x = 0, the redistribution weights are undefined.Source: F LRP-HALF: CONSERVATION AND… · p. 18, §F.1 · p. 18

F.2 FACTORIZATION DEPENDENCE AND THE GRI COUNTEREXAMPLE

The specified LRP-half baseline can produce different relevance allocations for equivalent radial factorizations and can fail zero-curvature consistency. The section also distinguishes origin-centered radial scaling from bias operations and discusses conservation under numerical stabilization.

F.2 FACTORIZATION DEPENDENCE AND THE GRI COUNTEREXAMPLE

  • Equivalent radial paths can yield different coordinate-wise relevance at the same input despite conserving the same total, giving an LRP-half counterexample to GRI.The direct and Lorentz paths differ in how their radial modules factor the operation.Source: §F.2 · p. 19

F.3 FAILURE OF ZERO-CURVATURE CONSISTENCY

  • As curvature approaches zero, the specified LRP-half rule continues redistributing relevance through the radial branch, so it fails zero-curvature consistency for both origin Poincaré maps.The same argument applies to the origin exponential map.Source: §F.3 · p. 20

G.1 POINCARÉ BIAS OPERATION

  • Möbius bias addition is not an origin-centered radial scaling because its input-dependent bias contribution generally changes the output direction.Although the bias is fixed, both coefficients depend on the input.Source: §G.1 · p. 20

G.2 SIGNAL-ONLY RELEVANCE CONVENTION

  • The signal-only relevance convention is proposed as a conservative way to explain predictions in terms of input features rather than fixed model parameters.The passage distinguishes this attribution convention from a consequence of the radial factorization result.Source: §G.2 · p. 20

G.3 MATCHED LORENTZ REALIZATION

  • The matched Poincaré and Lorentz bias realizations implement the same operation in different geometric representations.The Lorentz realization is constructed through the Poincaré–Lorentz isometry.Source: §G.3 · p. 20

H TANGENT-SPACE PROPAGATION AND NUMERICAL STABILIZATION

  • Tangent-space LRP-0 conserves relevance under the stated nonzero-output condition, while numerical stabilization can introduce a residual that must vanish or be handled for exact conservation.NRM is given as one example of explicitly handling the residual.Source: H TANGENT-SPACE PROPAGATION AND… · p. 21

I GEOMETRIC REPRESENTATION INVARIANCE EXPERIMENT

The geometric representation experiment compares relevance at the common input for equivalent Poincaré and aligned Lorentz paths. A separate sEEG perturbation test compares relevance-guided time-window removal with random removal.

I GEOMETRIC REPRESENTATION INVARIANCE EXPERIMENT

  • LRP-radial-all gives identical input relevance across the two equivalent paths, whereas LRP-half gives different allocations despite both summing to the target score.Table 3 reports relevance at the common original input for the two logarithmic-map paths.Source: I GEOMETRIC REPRESENTATION INVARIANCE… · p. 22, Table 3 · p. 22

J VALIDATE SEEG RELEVANT WINDOW EXPLANATION BY PERTURBATION

K MODEL CONFIGURATION FOR CIFAR-10

The CIFAR-10 configuration uses a custom lightweight Lorentz convolutional network and evaluates attribution methods with pixel perturbations and fidelity-sparsity curves.

K MODEL CONFIGURATION FOR CIFAR-10

  • Normalized RGB inputs are mapped through an origin exponential map using a 0.25 input scale to moderate initial tangent-space radius and hyperbolic coordinate magnitudes.The model fixes curvature to −1.Source: K MODEL CONFIGURATION FOR CIFAR-10 · pp. 22–23
  • The custom network uses six 3×3 convolutional layers, averages spatial coordinates over the final 8 × 8 grid, and classifies in tangent space rather than using a Lorentz centroid.The channel widths are (32, 32, 64, 64, 128, 128), with two convolutional layers at each of three feature resolutions.Source: K MODEL CONFIGURATION FOR CIFAR-10 · p. 23
  • The selected checkpoint reaches 85.70% test accuracy after selection by validation accuracy, and attribution evaluation uses 512 test images with 95% bootstrap confidence intervals.Integrated Gradients uses a zero baseline and 128 integration steps; sparsity is evaluated in 5% increments.Source: K MODEL CONFIGURATION FOR CIFAR-10 · pp. 23–24
  • The relevance procedure redistributes convolutional relevance across spatial and time-coordinate branches, then uses squared-coordinate proportions because each patch time coordinate is determined by its spatial patch.The first layer uses zB; subsequent layers use the γ-rule, and the same time-coordinate rule is used for both radial-map methods.Source: K MODEL CONFIGURATION FOR CIFAR-10 · pp. 23–24
  • Zero patches, affine biases, and numerical stabilization contribute to the recorded total relevance residual.A zero patch’s time-coordinate relevance is recorded as unassigned.Source: K MODEL CONFIGURATION FOR CIFAR-10 · p. 24

L ADDITIONAL FIGURES FOR EXPERIMENTS

The additional figures present qualitative attribution comparisons for CIFAR-10 and averaged sEEG signals with relevance heatmaps for two working-memory-load classes.

L ADDITIONAL FIGURES FOR EXPERIMENTS

  • Figure 10 compares input images and attribution heatmaps from LRP-radial-all, LRP-half, Gradient×Input, and Integrated Gradients on four CIFAR-10 examples.RGB-channel attributions are summed; red and blue denote positive and negative values.Source: Figure 10 · p. 25
  • Figure 11 shows averaged signals and relevance heatmaps for low- and high-working-memory-load classes, with each time series representing a sensor.Red and blue mark positive and negative relevance for the explained class score.Source: Figure 11 · p. 25, L ADDITIONAL FIGURES FOR EXPERIMENTS · p. 25

Related papers

Loading 2609.35128v1…